Rigid pool sizes?

OpenZFS
1

It looks like it’s important to get your vdev and pool sizes correct up front. It looks hard to muck about with those after the fact.

Is that a fair assumption?

2

IMO only partly true.

It is definitely easier / better to have a pool layout and stick with it, but there is a reasonable amount of flexibility if you need it i.e. you can add and remove mirrors or vDevs from an all-mirror pool, and you can add a drive to a RAIDZ vDev to expand the space. But there are some minor consequences and / or follow-up actions that you might need to do.

3

Could you please elaborate on this, thank you.

4

I am not a true expert here but a few examples that I understand to be the case (but I may be wrong):

  • When resilvering you should try to add a drive to your hardware, resilver and then remove the old drive because that retains more redundancy during the resilver.
  • When you remove a mirror vDev, blocks are mapped rather than moved - and the mapping takes some resources.
  • When you add a new mirror or RAIDZ vDev, it remains empty and you need to run zfs rewrite to rebalance the pool.
  • When you expand a RAIDZ vDev, although data is moved around to spread it evenly across the new number of drives, the for existing data the old data:parity ratio is retained and you need to run zfs rewrite to make the data:parity match the new layout (and recover some space).
5

Since OpenZFS 2.3 you’ve been able to expand a vdev by adding drives. What you can’t change though is the vdev’s parity level. So if you have a raidz1 vdev, you can’t increase the parity to raidz2 by adding drives or by any other means. So that’s an important thing to decide when creating a vdev.

You can also expand a pool’s capacity by adding more vdevs later on. Eventually a vdev becomes “too large” in terms of number of disks. There have been many discussions around when that point is.