Which Linux distribution works best with OpenZFS (and ZFS Boot Menu) and allows easy OS upgrades?

I’m currently running a Debian Bookworm distribution (MX Linux) and am considering the future and which operating systems could support OpenZFS and the ZFS Boot Menu. The clear choice would be to stick with Debian, but I’m interested in exploring other options.

Ubuntu offers built-in support for encrypted ZFS volumes, which I believe is crucial for a laptop. However, I’ve come across information suggesting that Ubuntu’s customizations and setup of OpenZFS are suboptimal.

Given the desire to upgrade the OS over time without needing to carry out a full reinstall with each major release, what are people in this community using (for single laptop single SSD), taking these factors into account?

Jim: If you read this, you can blame 2.5 admins for this question. It sounds like you guys have this running without a hitch on your laptops. Installing from scratch seems like a doable thing, MIGRATING there from a current installation running ext4 seems like a tough hill to climb. Thoughts?

Thanks!

1 Like

Ubuntu is the easy answer, by far.

Most of the “suboptimal” stuff in Canonical’s ZFS packaging revolves around Canonical’s own ZFS-on-root system, zsys, which never really got first-class treatment (read: actually finishing it). If you’re using ZFSBootMenu, you won’t get exposed to any of that, so it’s not a big issue.

I will note that Canonical has also been guilty of not coordinating as closely as it could with OpenZFS upstream, which has led at least once to them pushing out a build based on an already-obsolete upstream version with a critical bug that users really shouldn’t have been exposed to at all. But honestly, I’m not gonna pillory them on that; these things happen every now and then, and I’m not going to scream at Canonical any harder for pushing out a package based on a not-great version number than I am screaming at the OpenZFS devs for missing some issues in a particular version number and needing to correct them hastily in the next.

7 Likes

Debian Stable or Ubuntu Long Term Support (LTS) will be my choice for critical infrastructure such as your storage.

  • Reliable
  • Up-gradable with less or not issues
  • Great community support

Hope that helps,

I migrated a working Fedora 40 (ext4) laptop to a fresh encrypted ZFSBootMenu just following their guide with three minor changes:

  • When it came to installing the system, instead of rsync from the live-base I mounted the old ext4 partions for / and /home and rsync’ed from those into the ZFS datasets
  • I used systemd-boot instead of either the direct or rEFInd options but that is only because the laptop I was using (a system76 with coreboot) doesn’t like options outside of systemd-boot. Which can be done following these steps with minor adjustments for your distro of choice
  • With Fedora specifically I had manually copy the kernel to /boot after dracut and rename it to match it’s version i.e. /boot/vmlinuz-6.8.9-300.fc40.x86_64

So may sound like a lot, but it’s actually super painless and upgrades have been a non-issue thus far (I’ve been using ZFS on CentOS machines for a very long time and only had a small handful of upgrade issues over the years).

4 Likes

Thank you for sharing that experience! I’ve been getting a bit curious about the idea of live-migration of existing installations recently, nice to hear you got one working without too much hassle.

1 Like

Hi, I did try to follow the ZFSBootMenu installation tutorial for Ubuntu Jammy, but with the “noble” release (here - Jammy (22.04) UEFI — ZFSBootMenu 2.3.0 documentation).
Installation works but cant get the desktop environment to work?? Any ideas what Im doing wrong here?
Changed all Jammy to noble of course…

apt install ubuntu-desktop is all that takes.

3 Likes

Thanks, but I must have messed up something else…
During installation I noticed that I couldn’t access the ubuntu-archive-server and when trying to fix the failure in name resolution, I couldn’t edit my “resolve.conf”. Will set up a separate thread here next time Ill dig in to this. Thanks again!

1 Like

I’ve been doing a couple of ubuntu 24.04 installs with zfsbootmenu recently. For a desktop, I had to apt install ubuntu-desktop-minimal before I did the exit of the chroot and the reboot. And then everything came up fine.

For a server box, I had to configure the network using netplan, otherwise the ethernet did not come up - see another recent post i made about that. I also did apt install ubuntu-server-minimal before exiting the chroot.

1 Like

I might be remembering incorrectly, but wasn’t there some considerations for Debian on ZFSBM and ZFS, vs Ubuntu on ZFSBM?

It essentially boils down to “if you actively PREFER zfs-dkms, Debian gets in its way much less than Ubuntu does.”

The flaw with zfs-dkms is DKMS itself. The feature is that zfs-dkms is direct from the openzfs team, not filtered through Canonical’s maintainers, who have occasionally frustrated people with their release schedule (in one case, they released an upstream version with a pretty obnoxious bug, which the openzfs team fixed with a new minor release almost immediately, but Canonical neither packaged the new minor release, nor backported the fix from upstream into the version they HAD released).

2 Likes

Just my 2 cents here, but for Ubuntu vs Debian, I guess it might also depend on your use case. For a server, the tradeoffs Jim mentioned are relevant. Since you mentioned laptops, it might also be worth considering how much you like Snaps. It seems Ubuntu is going all-in on Snaps for user-facing software (eg. web browsers) and transitioning more and more packages to use this. If you like snaps, this is a feature. If, like me, you don’t, then this is a significant drawback.

3 Likes

cachyOS a possible option:

  • it has built zfs modules in it’s repos
  • a mirrored zfs root is quite easy
  • rolling release should never need to re-installed

I build my own kernels with signed zfs / other out-of-tree modules with arch-sign-modules. I usually just run linux-lts & linux-hardened kernels to always have a zfs capable system

This has been fine for about 8 years with zfs-dkms. If the latest kernel doesn’t build - I just run linux-lts for a while or more usually stay on an older kernel temporarily until zfs development catches up to current stable

Hey, I’ve been reading a lot of documentation as I’m about to deploy a home NAS and I decided that as learning experience I wanted to deploy ZFS on linux directly and not TrueNas.

Would you go for zfsbootmenu, or Follow the instructions from OPENZFS.
The computer I inherited for my Nas Build has 2 ssd and 3HD. I does any one have a good guide for zfsbootmenu on mirrored drives?

Thanks

If you want ZFS on root–which you do NOT actually NEED for a NAS–I’d go the ZFSBootMenu route.

Alternately, you could simply boot from an ext4 root and put all the stuff that matters on your pool. That’s still what I most frequently do, personally.

2 Likes

That was my first idea. but since The box came with two 512GB ssd I though. I could use it to learn ZFS on root + I’d get a bit of benefit from the mirrored ssd.

I’ll look for some ZFSBootMenu that covers mirror boot.

Thanks

I know this isn’t exactly OpenZFS but there is a free to us Community edition of Solaris ( OS) OmniOS that you can use to run ZFS (Oracle zfs) which will basically bullet proof.

I had similar problems while following those instructions. Had no effective network when I booted. I suspect I missed a step. The second time I made sure I had the ubuntu-minimal meta package MetaPackages - Community Help Wiki installed during the chroot stage. It was already installed - convincing me I had missed a step. The network was good and repositories accessible, when next I booted the new system. Using the new free VMware workstation 17.6 is a great way to try everything out.

I am just using ext4 on boot drive, which does nothing else than that in my server. Then additional disks as ZFS; I feel it’s cleaner having my “NAS data” separate from OS data.