Hey folks, just discovered this awesome board and wanted to pick your brains on this.
I’ll quickly describe our current situation and what we plan on replacing this with, because we’re running out of space.
We operate a backup server (15x 10TB HDD + 1x 1TB, 256GB RAM) with a mixed workload. This consists of about 50% incremental zfs receives for datasets between 10 and 5000GB (increments with up to 10% of data changed between each run) and 50% rsync/hardlink based backup tasks (rarely more than 5% of data changes between each run). So from how I understand the underlying technical aspects behind these, about half the workload is sequential writes (zfs receive) and the other half is a mix of random/sequential read/write tasks.
Since this is a backup server, most (not all) tasks run at night and often from multiple systems (5-10, sometimes more) to backup in parallel.
Our current topology is a 5x3way mirror with one SSD for L2ARC:
`config:
NAME STATE READ WRITE CKSUM
s4data1 ONLINE 0 0 0
10353296316124834712 ONLINE 0 0 0
6844352922258942112 ONLINE 0 0 0
13393143071587433365 ONLINE 0 0 0
5039784668976522357 ONLINE 0 0 0
4555904949840865568 ONLINE 0 0 0
3776014560724186194 ONLINE 0 0 0
6941971221496434455 ONLINE 0 0 0
2899503248208223220 ONLINE 0 0 0
6309396260461664245 ONLINE 0 0 0
4715506447059101603 ONLINE 0 0 0
15316416647831714536 ONLINE 0 0 0
512848727758545887 ONLINE 0 0 0
13087791347406032565 ONLINE 0 0 0
3932670306613953400 ONLINE 0 0 0
11052391969475819151 ONLINE 0 0 0
2750048228860317720 ONLINE 0 0 0
17997828072487912265 ONLINE 0 0 0
9069011156420409673 ONLINE 0 0 0
17165660823414136129 ONLINE 0 0 0
4931486937105135239 ONLINE 0 0 0
cache
15915784226531161242 ONLINE 0 0 0
`
We chose this topology (3 way mirrors) because our main fear whats losing the whole pool if we lost a device while reslivering (which actually happened TWICE in the past 4 years). But we sacrifice so much storage space here and are not super sure if this layout actually offers a decent performance for our specific workload.
So now, we need to replace this system. Our only option (sadly) is to use a server with 14x 20TB HDD and 2x 8TB SSD drive configuration. We get 256GB RAM and some 32 core CPU monster.
Since we do not have access to 15 HDDs, we cannot simply reuse the configuration and maybe it’s not a bad idea to reevaluate our setup anyway.
Although this IS only a backup maschine, losing some 100TB Pool and Backups from ~40 Servers, some going back years, is not something we want to experience. So we need to atleast sustain double drive failures (we’re constantly monitoring) or a drive failure during resilver.
Now, what ZFS Pool setup would you recommend for the replacement system?
How can we best leverage these two huge 8TB SSDs?