I know we’ve talked in various threads here about what counts as a sane place to start for setting up ZFS snapshot schedules for various specific workloads workloads (like Time Machine), but overall it feels like one of those semi-overwhelming things that’s going to take experience to get right–and attempting to learn to get it right as you go might end up resulting in data loss.
We already have a great set of recommendations for sane defaults on recordsize and volblocksize, and they really help me feel confident just doing things with my NAS.
But I’ve been hesitant to set up snapshots or even start some projects that I want to use snapshots with because I feel like I don’t know what I’m doing and don’t want to waste CPU cycles and storage space (and increase power/heat) on my NAS by taking too many or retaining them too long.
For me, at least, it feels like there’s at least a few different categories of snapshot that would need their own strategy (scheduling and retention):
- Datasets used to back up production workstations, (e.g., office work, software development, etc.–the things we use to make money), including user data and system configurations that need to be restored in the event of a problem. EDIT: Tools I’m planning to test/use:
- Restic
- Proxmox Backup Server
- Carbon Copy Cloner and SuperDuper (Mac backup utilities)
- Rsync/rclone
- Self-hosted internal services data, including configurations. I feel like this snapshot strategy would be a bit more aggressive in frequency and maybe somewhat less aggressive in retention, as the data would change more, faster, and keeping older snapshots might not be as useful.
- Externally-exposed services data, (e.g., websites, Minecraft servers, etc.). Similar to (2), but I’m thinking you’d want to be a bit more aggressive on frequency and retention because something exposed to the outside world is more likely to get smoked.
- Media and other large file read-only datasets (e.g., vintage game ROMs, your family’s precious videos and photos, etc.). As long as you’ve got an up-to-date, snapshot index of your legally obtained archives, I don’t think it makes much sense to go aggro on snapshotting data that rarely changes. The metadata would change quite a bit over time, and I’d want to try to preserve that, but I’m not going to be that upset if something happens to the last 24 hours of progress I made in my 40th play through of Super Metroid.
- A/V production. This is not an area I do much of anything in, but I put it on the list because I suspect that the snapshot strategy needs of someone doing real work on large, changing files is going to be different than any of the above.