3-2-1 Backup of ZFS in a Home Server Environment

OpenZFS
1

Hi, I’m Maran, and I’m new to this forum. I’ve already found some useful information here, which is why I decided to create a post of my own. I don’t work in the storage industry, but I’m tech-savvy. It’s a hobby of mine, as well as a way for me to avoid Big Tech. I like to experiment with technology. Due to time constraints, I look for solutions that require some tinkering or attention but can work for six months without daily maintenance during stressful periods when I can’t devote much time to it.

I run my own home server and I really like it. I use TrueNAS CE for this purpose because it provides the power of ZFS through a user-friendly web interface. I like my home server so much that I started adding valuable things to it, like family pictures, which brings up the question of a sound backup strategy.

I already take advantage of ZFS’s incredible replication functionality to stream my data to a friend’s home far away, in a different country. This way, I have a second off-site copy of my data. However, I have three issues with my setup:

  1. I still need a third copy.
  2. I need a medium other than ZFS.
  3. My friend’s Internet is so slow that backups work fine over time, but in a serious recovery scenario, it would most likely be cheaper to catch a flight and transfer the data locally. This is kind of okay for one backup copy, but without a third, fast copy, it’s kind of unacceptable.

In order to implement a decent 3-2-1 backup strategy, I would need an additional medium. Off-site storage is already covered. Please let me know if I am wrong or have overlooked something.

Personally, I thought it would be best to have another server in the same rack running two services.

  1. Proxmox Backup Server for my VMs
  2. Something that takes care of my ZFS data. I thought MinIO might work.

What would you recommend?

I’m looking for a relatively cost-effective solution. I don’t run a company; I just have a bunch of valuable family photos. It should also be able to be turned off so that I don’t have to pay for electricity all the time. Since my home server already has 80 TB, a multi-disk setup with MinIO would be a feasible way to achieve resiliency.

The idea is that the backup system will lock every file for three months. Additionally, the backup solution should be fast. Nothing extreme, but the backup and recovery times shouldn’t be excessive, as would be the case if I needed to use data from my friend’s ZFS replication box. However, I would be happy if I maxed out at 1 Gbps.

Initially, I planned to set up a second server with Proxmox and virtualize both Proxmox Backup Server and a VM running MinIO with attached disks. Any thoughts? I might also add SSD/NVMe disks to the Proxmox Backup Server VM to speed up VM backups. Then, I could load all the data onto my main home server and stream it to my friend’s TrueNAS system, as well as to the time-locked system next to my main system. All of this should be fully automatic. Additionally, I would like to back up my Proxmox VMs in case something breaks, so I can recover quickly.

Due to price and privacy concerns, as well as my desire for relatively fast recovery times, which cannot be achieved with cloud solutions at my current location, I am not looking for another cloud solution (I consider my friend’s replication box to be one). My Internet speed is nowhere near 1 Gbps, whereas 1 Gbps is typical for an inexpensive Ethernet cable when only local networking is involved.

I hope to take advantage of the storage expertise on this forum and get feedback about my plan. Please tell me if I’m on the right track or suggest better alternatives. Any other feedback is much appreciated!

2

I use MinIO as a backend for restic and autorestic. I target both a Backblaze B2 bucket, and a locally-hosted MinIO bucket. Works quite well, although I sometimes run into issues where the bucket is locked and restic craps out until I go run restic unlock manually. Some monitoring there is a good idea, if you go that route.

MinIO recently had some controversy and removed some nice admin UI features. Since you mentioned in your post you like TrueNAS because of a “user-friendly web interface” you would be stuck using the mc command (or buying a commercial license with MinIO to access some other web GUI they have for administration).

Since it’s photos you’re backing up, I’ll mention Immich which is self-hosted and you could setup to run on a separate system.

As an iPhone user, I personally use icloud_photos_downloader to grab photos from iCloud, store on ZFS, and shuffle off to B2 and MinIO via restic.

I run a second box with Proxmox Backup Server and I agree it’s a nice safety net for VM backup and restoring is dead simple. (Although that’s just the VM, underlying datasets that I mount as Samba shares in the VM aren’t backed up to PBS.). Highly recommend using PBS if you’re in the Proxmox ecosystem.

2 Likes
3

I’m using TrueNas Core for storing my personal data. Next to zfs replication to an offside location I use Borg Backup. The software is open source and free. The reason I use this next to zfs replication is it is cheaper to buy target storage space. If I want to replicate to a zfs host then I have to rent a VM (or rent zfs space), the Borg Backup target doesn’t have to be an vm with zfs, it requires ordinary space which is cheaper.

Borg backup doesn’t rely on zfs replication basics, it supports data deduplication and data encryption. I run it in a separate jail as non root user. I created a backup user which should be able to acces the data to be backuped. I like its speed, if data to be backuped doesn’t change then backups are fast. Next to an automated backup job, I do automated tests to assert some files contain an expected checksum. The backup an test are assured to be run by triggering healthchecks.io. If jobs fail or aren’t run at predefined frequencies then I will be informed.

1 Like
4

Thank you for your insight!

I want to back up data other than images, too. It was only the images that prompted me to look into decent backup options.

I am familiar with Restic and Borg.

I know Proxmox Backup Server only backs up VM data, not attached data.

What is your recommended method for mounting data into individual Proxmox VMs from a TrueNAS VM on the same physical machine?

Before we discuss specifics, I would like to know if it makes sense to back up ZFS data on another medium. If so, which medium would you prefer? Would it be S3 with Restic or Borg?

Can Borg or Restic be integrated directly into TrueNAS?

What about the TrueNAS “Cloud Sync Tasks” tool in the web interface? I can set up Amazon S3 as a target, which should allow me to point it to my MinIO server. Would that be a good option? Why use Borg or Restic instead of an integrated feature? Or am I misunderstanding what TrueNAS Cloud Sync Tasks are?

Thanks!