This is a followup to this thread: Understanding Relationship Between zVol Block Size and iSCSI Logical Block
tl;dr, while @mercenary_sysadmin explains in that thread exactly why a 4K LBA extent block size is not ideal…
But more importantly, 4KiB clusters eat a LOT of IOPS. You rarely want this in a VM, for the same reason you rarely (read: damn near never) want 4KiB clusters / blocks / volblocks for a VM using 4KiB native sectors: very few workloads actually fragment data that heavily, not even databases.
Also–and again, much like ext4–data under ntfs is stored primarily in extents, not just clusters. Extents are a range of contiguous clusters which are read or written in a single IOP. These tend to average closer to 64KiB.
Even Microsoft SQL server typically defaults to 64KiB extents: Pages and Extents Architecture Guide - SQL Server | Microsoft Learn
This means that you generally want to match your block size, or volblocksize, to roughly match the typical extent size, not cluster size. So, 64KiB.
…TrueNAS’ dev team isn’t budging on making bigger extent sizes an option at this point, and there’s not really a way to override that at the CLI level. So, while TrueNAS’ dev team might be willing to look at experimental benchmarks showing the performance difference, there’s no easy way to get those test results if all you have available is a TrueNAS system.
I was wondering if anyone knew of any benchmarks that are publicly available doing A/B testing of otherwise similar iSCSI setups where the only thing that changes is the extent block size.
I’ve been using 64k volblocksize zVols with 4K extent block size iSCSI targets in TrueNAS since I’m using the iSCSI targets for VM data storage and QEMU/KVM likes 64k volblocksize, but I have no idea if that’s actually the best way to go when the extent block size is constrained to 4K. None of the “How to iSCSI with VM backing storage” tutorials I look at really talk about the volblocksize/extent block size relationship. At all.
EDIT: I should clarify that I’m currently interested in non-boot data storage. Like a Steam cache, or any other large data store that shouldn’t be on the VM’s boot virtual disk. I want to try ExoDOS in a Windows VM, but I don’t want to store hundreds of GB of DOS games in the boot zVol. :P.