My workstation was broken and neglected for a few weeks and some of the the filesystems on my NAS rolled out of the similar-snapshot window. I’ve heard that bookmarks could help ameliorate this condition, but that’s something I’ve never explored. Can someone explain if this could have helped?
I know the question says sanoid, but I’ll answer using syncoid. There are things in syncoid that would have helped in your situation. Please ignore this if you are not using syncoid together with sanoid.
For syncoid, if you use the --create-bookmark together with --no-sync-snap, whenever it syncs, it creates a bookmark of the newest snapshot. This way, any snapshots created after the bookmark can be incrementally synced.
Alternatively if you don’t care too much about the space savings of bookmarks, you can use the --use-hold flag in syncoid. This will prevent your latest synced snapshot from being deleted. The --use-hold flag will release previous holds after it creates a new hold.
1 Like
Those are good things to try, thanks.
In case it is helpful, I thought I would briefly explain.
Suppose you have snapshot A on machine 1. You transfer snapshot A to machine 2. You later take snapshot B on machine 1, and you delete snapshot A on machine 1. You want to transfer the incremental from A-B to machine 2. You don’t actually need a copy of the full data in snapshot A. You only need to remember the exact time point of snapshot A. Then, you can just transfer all of the blocks that changed after time point A (up to B). That’s what a bookmark does. It just remembers a specific time point without needing to retain the data content.
Bookmarks are only useful in a single scenario:
When you already have a common snapshot between source and target–note, you must already have the common snapshot on both sides–you may first create a bookmark on the source and then destroy the snapshot on the source without breaking the replication chain.
However, the snapshot still has to be present in full on the target.
What a snapshot actually is is a collection of blocks. What a bookmark actually is is the birth time of the most recently written block in the snapshot it was created from.
What allows you to incrementally replicate from a bookmark is the fact that the target already has all the actual blocks necessary, so the source doesn’t need to know what’s in those blocks in order to incrementally replicate–but it does need to know, VERY precisely, what starting point to use.
So, if you’ve got bookmark #1 still available on the source, and snapshot @1 present on the target, you can create @2 on the source and replicate incrementally, because ZFS knows it just needs to get rid of any blocks on the target newer than the birth date recorded in the bookmark, then fling over any blocks contained in @2 that are newer than the birth date recorded in #1.
This allows you to more easily keep much shallower archive depth on a source system with a few (or one) small, fast drives, while keeping much deeper archives on a target system with many more, or larger drives.
In the most extreme example, you can get away with keeping no snapshots on the source at all–you create a snapshot @1 on the source, bookmark it as #1, then do a full replication to the target.
Once that full replication finishes, you can destroy the snapshot on the source. Next time you need to replicate, you take a snapshot @2 on the source, bookmark it as #2, then replicate incrementally from #1 to @2. Once that incremental replication finishes, you can then destroy the snapshot @2 on the source, leaving the bookmark #2.
It keeps working the same way after that. (It also works the same way whether we’re discussing manually taken snapshots and bookmarks and replication, or whether we’re using sanoid and syncoid.)
1 Like