As the title says… I run a couple of VMs (QEMU/KVM) with disk images as QCOW2 files stored on ZFS. But I don’t really know which options that are optimal, and haven’t been able to find some clear advice when googling.
I usually use the same -o cluster_size=XXX as the ZFS recordsize, -o preallocation=metadata since it (allegedly) makes image size increases quicker.
But what about the -o nocow option? It seems to me that “double copy-on-write” (in both QCOW2 and ZFS) is downright bad, but I have not found any ZFS-specific data on this topic.
And while we are talking about QCOW2 options in general, what about -o lazy_refcounts/-o extended_l2/-o compression_type/-o refcount_bits?
I have created a ZFS benchmark bash script, based on fio and a Jim Salter Ars Technica article on this subject from a few years ago, which I recently modified to also run fio tests on QCOW2 files (by means of an ext4 filesystem created on a this container after mounting it using qemu-nbd), but obviously the performance will depend on how well QCOW2 and ZFS play together… It still needs a bit of polish, but choosing the “theoretically” best qemu-img options would be a good start.