I have a fairly simple ZFS setup: one zpool, two mirror VDEVs, each consisting of two 4 TB WD Red drives. I have purchased four new WD Gold 14 TB drives. My question is, how best to add these drives to my zpool. I have thought of two options:
(1) create two new mirror VDEVs, each with two of the 14 TB drives. Effective storage would be ~8 TB from the existing VDEVs and ~28 TB from the two new VDEVs
(2) create one new mirror VDEV using two of the 14 TB drives. Save the other two new drives for building a second ZFS server as a remote backup target. Effective storage: ~8 TB from the existing VDEVs plus 14 TB from the new VDEV.
How should I be thinking about this? What are the criteria for designing an optimal setup?
You might want to consider aging the 4T drives out–zpool replace them with the 14T drives and call it a day. How old are they? How prepared are you to replace them in a hurry after one dies?
I’m not saying you have to sunset your older 4T drives now, mind you. But when the new drives you bring home are triple the size of the ones you already had, it is at least time to consider it.
They are not that old. I bought them when I build the server in 2023. I would hope that the useful life of a set of drives would be longer than three years. As for replacing them in a hurry, this is a hobby server, not a production system in any sense. My plan for replacing one of them is to drive to Microcenter, pick up a drive, and come home and swap it in. If it takes a couple of days, I can live with that.
Fair enough! Odds are that you’ll get another three to four years out of them, then.
In that case, I’d recommend going with your plan for creating a backup server–except I’d advise the 14T pool to be source, and the 22T pool to be the target (backup) rather than the other way around.
If you have time, and don’t need the space right away…
Current setup:
Main pool: (mirror 4TB 4TB) (mirror 4TB 4TB)
Stage 1:
Replace one 4TB in each of your mirrors with a 14TB.
Create a backup pool with the same setup.
Main pool: (mirror 14TB 4TB) (mirror 14TB 4TB)
Backup pool: (mirror 4TB 14TB) (mirror 4TB 14TB)
Effective space at this point: 8TB in main pool, 8TB in backup.
Run in stage 1 for a few months, run extra scrubs if you can to build confidence in the 14TB drives.
Stage 2:
Remove two 14TB from your backup pool, swap them with the 4TBs.
Take the 4TBs that you took out from your main pool, and put them non mirrored.
Main pool: (mirror 14TB 4TB) (mirror 14TB 4TB)
Backup pool: (4TB) (4TB) (4TB) (4TB)
Effective storage: 28TB in your main pool, 16TB in your backup pool.
