ZFS Pool Audit & Upgrade Path Advice

Hey all,

Over years of homelabbing, I’ve built a “lab of theseus” upgrading and swapping parts piecemeal (and probably impulsively). That worked fine back when gear was dirt cheap, but today’s market has forced me to rethink my strategy and plan purchases more deliberately.

Right now, my storage setup is the biggest bottleneck. I’ve qualitatively noticed some operatings are sluggish (extractions, metadata queries etc) probably likely due to my pool over-provisioned containers, so I want to optimize what I already have, stretch my budget, and map out a smarter long-term upgrade path.

I’d love a once-over on my storage architecture and any specific advice on areas where I can improve would be greatly appreciate


Pools:

  1. hot
  • 2x 1TB Samsung 870 EVO SATA SSDs in a single vdev stripe
  • Hosts the root dataset and container datasets. I understand I am at risk of data loss, but wanted to get as much preformance as possible. This is acceptable to me as I preform hourly ZFS local replication snapshots sent to bulk pool.
  1. cold
  • 3-vdev mirror using used Seagate Exos.
  • Stores bulk media datasets, and receives replicated snapshots from the hot pool.
  1. scratch
  • 1x 250GB Samsung 870 EVO SATA SSD
  • Serves as an active torrent download dataset. Completed downloads are moved to the cold zpool and are seeded directly from this pool.

Considering the following; adding a super duper old 256GB WD Black PCIe SSD to the cold pool for cache, and a new Samsung SSD 990 PRO to take over the hot pool and figuring out what to do with those free SSDs later

Thanks!

zpool list
NAME                                                    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
scratch                                                 232G  75.8M   232G        -         -    28%     0%  1.00x    ONLINE  -
  ata-Samsung_SSD_870_EVO_250GB_S6PDNZ0W307889N   232G  75.8M   232G        -         -    28%  0.03%      -    ONLINE
cold                                                   45.5T  25.4T  20.1T        -         -     8%    55%  1.00x    ONLINE  -
  mirror-0                                             16.4T  8.95T  7.40T        -         -     8%  54.7%      -    ONLINE
    ata-ST18000NM000J-2TV103_ZR51GNT3                  16.4T      -      -        -         -      -      -      -    ONLINE
    ata-ST18000NM000J-2TV103_ZR52W9BZ                  16.4T      -      -        -         -      -      -      -    ONLINE
  mirror-1                                             14.5T  8.23T  6.32T        -         -     9%  56.6%      -    ONLINE
    ata-ST16000NM001G-2KK103_ZL2A584X                  14.6T      -      -        -         -      -      -      -    ONLINE
    ata-ST16000NM001G-2KK103_ZL2AACJE                  14.6T      -      -        -         -      -      -      -    ONLINE
  mirror-2                                             14.5T  8.20T  6.35T        -         -     9%  56.3%      -    ONLINE
    ata-ST16000NM001G-2KK103_ZL2A1FCG                  14.6T      -      -        -         -      -      -      -    ONLINE
    ata-ST16000NM001G-2KK103_ZL2ASG10                  14.6T      -      -        -         -      -      -      -    ONLINE
hot                                                  1.81T   445G  1.38T        -         -    48%    23%  1.00x    ONLINE  -
  ata-Samsung_SSD_870_EVO_1TB_S75BNS0W405275E           932G   314G   616G        -         -    54%  33.7%      -    ONLINE
  ata-Samsung_SSD_870_EVO_1TB_S75BNS0W405272R           932G   131G   797G        -         -    43%  14.1%      -    ONLINE