Expanding a Z2 pool ... the hard way

OpenZFS
1

Hi there.
I’m totally new to ZFS.

I have been using 5x 8TB Seagate Ironwolf drives for years, in a slowly expanding collection using mergerfs and snapraid. Recently I bought a new drive, with the intention of finally going to ZFS Z2. I also had a Seagate archive drive on the side.

My challenge has been to preserve about 20TB (ish!) of data during the transition.
I can’t afford a “temporary” backup to stash all the data.

So by shuffling data around and sorting a fair bit of it, I’ve created a Z2 pool in OMV (and command line on Debian) using 4x 8TB Seagate drives (3x Ironwolf + 1 Archive).

Lets’ call this combination ABCX

My ignorant intention initially was to incrementally add the remaining 3 drives (DEF) as I copy data onto the pool, and remove the Archive drive (X) at some point … resulting in a 6x 8TB Z2.

But now that I’ve got ABCX going and ~5TB of data in the pool, I understand that I can’t add one drive at a time because of “mismatched replication level”? (this would be the ideal situation *1)

Instead, my next crazy idea would be to remove the archive drive (use it for temporary storage) to degrade the pool, and then add the remaining 3 drives PLUS the archive (DEFX) … and THEN remove the archive drive (X). (*2).

Are either *1 (using -f ‘force’?) or *2 even remotely possible?
How can I get to where I am now (ABCX) to where I want to be? (ABDEF running as Z2 and X in cupboard)

My PREFERRED option would be *1:
Incrementally adding the drives (D and E) with “-f”, if that’s possible, and then replacing the archive drive X with the remaining drive F.

*2 is going to be a BIG PITA, as I have to free up drives EF which are currently full.

Any suggestions?

2

Do you have any sort of backup for this data?

My recommendation would just be to rip the band-aid off and blow away the existing setup, then restore from backup onto a RAIDZ2.

RAIDZ isn’t really designed to be expanded one disk at a time (though RAIDZ expansion is finally a thing, it’s still not something I’d recommend doing if you can avoid it with proper planning. I also don’t think that feature has made it down to ZFS on linux yet.)

I’d also be careful with those Seagate Archive drives… if memory serves, they’re SMR, not CMR. Having that in your pool will likely tank your performance.

3

The backup(s) for some of this data has been spread across a fleet of older drives, including the Archive (yes, SMR) drive which is now being used in the hope of transitioning the pool (temporarily).

This is a situation limited by budget (or lack thereof!), so having a 20+TB “backup” really isn’t possible.

I think what I’ll have to do instead is start shuffling stuff around, create a pool with a Z1 vdev and copy data onto it, then if/when I can free up the remaining 3 drives subsequently add those as a 2nd Z1 vdev, and then resilver the lot.

1 Like
4

That’s a good plan. Three-wide Z1 is one of my favorite topologies. You’ll get significantly better performance and faster resilvers and a little bit better storage efficiency to compensate you for dropping from dual parity to single.