G’day,
I was just wondering what everyone’s thoughts were on using ZFS on root for a Ubuntu instance running KVM virtual machines. I know that @mercenary_sysadmin says that he still doesn’t use ZFS on root most of the time but I kind of like the idea of having the ability to rollback root if the host updates fail.
Thanks
I’m pretty much root-on-zfs everywhere - servers, workstations, laptops. Separate datasets for /home/<user>. I have an apt pre-snap hook that snapshots at least the root dataset, and anything else configured before any apt command modifies the system. That alone has saved systems.
Combine that with zfsbootmenu which makes it dead-simple to clone or rollback. It’s easy and safe to test things out, and you wind up being more willing to experiment knowing you can recover easily.
My own ZFS-root github repo builds the systems, and there’s a packer setup to create a qcow2 image that can be used directly in many VM environments. Even though ZFS is not directly managing the disk(s) there, the benefits of root-on-zfs outweigh that I think.
I moved on from Ubuntu a while ago, but under Proxmox I had a difficult time getting ZFS fully booting on root, with encryption. I switched to proxmox-boot-tool which uses systems-boot under the hood and it’s worked wonderfully. I expect Ubuntu has this sorted, but be sure to check if you want to do anything special like encryption or newer zfs features on the root pool.
For me, the big wins are snapshots via zrepl and compression. My root pool currently has a 1.32x compression ratio using lz4. Otherwise, the nice thing is that I’m using the same tool to manage the mirror and file system that I’m using on my data pools.
I’ve used snapshots to rollback VM images a few times. It’s nice and easy!
You could replicate all of this with lvm and other tools (and honestly may even be preferred for encryption), but it’s up to you how much you care!
/root on zfs is fine. But /boot (kernel+initramfs) should probably go on an ext4 partition to avoid the complications with GRUB, perhaps with encrypted zfs too.
If you keep /boot on rpool (not in its own bpool) look into ZBM (ZFS Boot Menu).
Though it seems the new hotness these days is UKI (Unified Kernel Image) stored in the EFI/ESP FAT32 partition that the UEFI firmware can verify the signature (secure boot) and boot directly, no need for Grub, ZBM or other boot manager complications. Unfortunately it doesn’t seem to be an official option in Ubuntu, though it doesn’t look hard to set up with just some minimal scripting. I haven’t used it but I’m curious whether I can make it work without any manual maintenance when the kernel is upgraded etc