I’m rediscovering ZFS and am very new to it. I’m currently using Proxmox (on an R730XD bare metal) + TrueNAS Core (also on an R730XD bare metal). I didn’t want to dive into the deep end by trying to get a bunch of VMs/containers and a storage array going on something like plain vanilla Debian 12 with just me and the shell. I figured that using Proxmox and TrueNAS Core just to get things going was a good idea, and then I could dig down to a more granular level. However, I am worried that the “pretty abstraction layer" that are Proxmox and TrueNAS may prevent me from learning “the best way" by providing too much hand holding.
If Proxmox and TrueNAS were out of the picture, what are some recommended “loadouts”? Was I close with vanilla Debian 12 and then something like virt-manager? Ubuntu server? GUIs a no-no?
Those of you who “roll your own”, what does that look like? Am I worrying too much about hand holding? Since you can obviously hit the shell on both, are Proxmox and TrueNAS fine?
I’m worried that using Proxmox and TrueNAS may not provide the best performance and may cripple my understanding if I don’t know what’s going on under the hood. Learning to do it all from scratch would give me the most knowledge, but it’ll definitely take a while! I’m just not sure what a good balance looks like.
I’ve been using Ubuntu server in the way that you’re describing for my file server and it has been a good experience. The support for ZFS is built-in (I’m not sure about Debian). On a separate host I also happen to run Proxmox, using ZFS as the underlying storage. Obviously I’m biased but this setup has allowed me (and I think would do the same for you) the best of both worlds. You learn about direct management of ZFS on the Ubuntu server as a NAS and enjoy the VM/container pretty UI features of Proxmox, while still having benefits of ZFS under-the-hood. I’ve been happy with the performance of both and I believe said performance has more to do with your pool hardware and configuration than anything.
As a side note, an simpler alternative to TrueNAS is Xigma NAS which is more purely a NAS distro and doesn’t have the jails/VMs that TrueNAS includes. It’s been recommended by Jim Salter on several podcast episodes.
Debian user here. Ubuntu has the advantage that ZFS kernel modules are built in so you just need to install the utilities. The instructions for installing ZFS on Debian are straightforward and easy to follow. The choice between the two comes down to whether you prefer Ubuntu or Debian. I suspect the same is true of many other distros as well.
Hi
Previous to my current setup - proxmox on a low power CWWK 5105 box - I ran debian stable on an old i7 dell. I setup KVM / libvirt. I setup everything - network bridging and storage using virsh and ssh to connect to host and all guests. This had the highest learning curve and was a good experience. Proxmox has more bells and whistles and ease of setup so I see your point. Perhaps using one of your servers for that and the other do a simulataneous slow build it doing it with straight debian?
Having been a long time CentOS/Fedora user, I made the jump to Debian in late 2020. (No points awarded for guessing what prompted that!) I dabbled briefly with Ubuntu, for the ‘easy’ ZFS integration but actually found Debian more stable and easier (for me) to use.
On top of ‘vanilla’ Debian (currently Bookworm) I have ZFS, of course, KVM for VMs, and Podman (can you believe it!) for containers.
It’s been awesome for me.
I do add Cockpit to my software load out on the system, which I like for a quick dashboard to check on how VMs are running, or console into a container, etc., Not required, but it makes a lot of things a little easier.
The only ‘non-standard’ package I put on there is the ZFS cockpit module from 45Drives. I don’t tend to use it to manage ZFS, but again, it’s a nice quick GUI to show me things. 45Drives Module
With Ansible (not, by any means, required), I can go from bare metal to fully configured in just a few minutes.
And, to be honest, I’m sad I didn’t make the swap to Debian years ago. It’s easier, and more reliable, for me.
Your setup that you described is what I would like to transition to. Currently running 2 TrueNAS Scale boxes, one primary and one backup. I have another Proxmox box running a couple VMs. Some of the features in TrueNAS Scale are nice, but feel a bit much and constricted at the same time. Would you mind sharing your Ansible playbook or point someone looking to learn in the right direction?
Thank you all for the feedback! I’m going to take some time and digest this, but it sounds like setting up a “dev” environment in my lab to experiment with this stuff is the best bet.
