G’day
I was just wondering if anyone had any suggestions for tuning ZFS for media storage specifically. I have have seen the excellent guide at ZFS tuning cheat sheet – JRS Systems: the blog on general tuning was just wondering if anything changes if you are only storing MP4’s.
Any help would be much appreciated.
Cheers,
Perhaps a good starting point is to get to know your MP3 database a bit better.
A useful command to run in the directory where you store your files is given by Wendell in his forum post:
find . -type f -print0 | xargs -0 ls -l | awk '{ n=int(log($5)/log(2)); if (n<10) { n=10; } size[n]++ } END { for (i in size) printf("%d %d\n", 2^i, size[i]) }' | sort -n | awk 'function human(x) { x[1]/=1024; if (x[1]>=1024) { x[2]++; human(x) } } { a[1]=$1; a[2]=0; human(a); printf("%3d%s: %6d\n", a[1],substr("kMGTEPYZ",a[2]+1,1),$2) }'
I think that can help you finetune your parameters.
only mp4
recordsize=1m is the only one that REALLY matters. Leave compression on; it won’t hurt performance much because it nopes out of incompressible data, and it will compress your slack space (which otherwise would increase with the large recordsize).
If you’re feeling really frisky, set compress=zle so the ONLY thing it tries to compress is your slack space. But this really isn’t likely to matter much vs compress=lz4 in any kind of real world way. Basically, do this if you’re itching to do everything as efficiently as possible, but don’t bother if you’re feeling overloaded already, or just don’t want more complications to keep track of.
On a similar note, atime=off, to save you a few IOPS is theoretically a great idea, in practice usually perfectly fine, but in this case won’t make enough difference to matter, because your workload bottlenecks on throughput, not IOPS–and if you’re serving it over a network, the NIC will be your true throughput bottleneck anyway.
So, yeah. Recordsize=1m is SUPER important for an all-mp4 workload; the rest is nice to have but probably won’t make a big difference to you one way or another.