Ive been playing with ZFS concepts for a while, but gradually moving to more real world usage of it, so I guess id still call myself a beginner.
To set the scene, im currently running a proof of concept in my homelab utilizing ZFS as the backing storage for an Ubuntu LXD environment, which allows for provisioning KVM based VMs and full OS containers. Default behaviour for spinning VM’s up here is to provision a ZVOL, and I dont believe QCOWs are a supported option.
My endgame is to have FreeBSD VMs running on this, so I have been trying to come up with some sound tuning options for this.
FreeBSD out of the box will support UFS and ZFS. Im leaning more toward UFS for the guest filesystem due to concerns about how nested ZFS would perform, but maybe this is unfounded, so input is welcome on that point.
Assuming I run with UFS for the guest, from reading man pages, it seems there are 2 tunables at the guest side: block size, and fragment size. Defaults out of the box are 4k fragment to 32k block, following a recommended 8:1 ratio. (referencing newfs(8) )
For the ZVOL, would I be looking to match up the volblocksize to the guest fragment size of 4k or block size of 32k for best results?
Also, ive read in a couple of places that primarycache=metadata can be beneficial on the host to allow guest OS to handle their own caching, does this still hold?
My test pool is a simple single disk nvme stripe, 4k native blocks so ashift=12. If all goes well the plan will be moving this to compute nodes with 2 x nvme mirror vdevs by default.
Thanks in advance for any feedback or advice on this one.