I have a server and laptop both running ZFS. The server is switched on 24x7, while laptop comes on frequently. I would like to keep a dataset synced between the laptop and the server. At the moment i have syncoid running to pull the dataset from the server on an hourly basis. I always copy any new files to the server and never to the laptop. This way i know that they are in sync. The server is running sanoid with production template and the laptop with backup template.
I am considering a change in strategy in which i can also write to the laptop. One idea i have is to run syncthing on the laptop and the server. So any writes to the dataset on the laptop will get copied over to the server automatically in the background. However, i am not sure how this ties with syncoid/sanoid.
I am curious what is the best way to keep laptop and server in sync?
ZFS replication will overwrite anything on the target dataset(s) that doesn’t exist in the source dataset(s), period, no ifs, ands, or buts. It’s not going to work for bidirectional sync of working data that may be changed at any time on either end.
It would be possible to clone the most recent dataset on the laptop and work on the clone without new replication in from the server messing with the clone. But that leaves you with the problem of getting stuff you did on the clone on the laptop back to the server, and also leaves you with no way to update the clone you’ve been using as a local working set with new data from the server.
Ultimately, bi-directional sync just isn’t compatible with ZFS replication. (It is also never, in my experience, at convenient as it sounds at first blush. You’re going to have issues with files that were independently modified on both ends, and you have to figure out how to reconcile them. But that’s just bi-directional sync in general, nothing to do with ZFS specifically.)
Ok. I guess i need to ditch the idea of using syncthing then. I guess I could send zfs dataset from laptop to the server. One reason i dont like that is because the laptop doesnt have ECC ram and has a single disk in contrast to server. Data integrity should be much better on the server than the laptop.
Given that, I would just use SyncThing to replicate changes from the laptop to the server, take periodic snapshots on both sides for safety, and back up the server data.
I’ve been fond of Zrepl for laptop syncs. My Lenovo has two nvme drives, one active, one a pure backup target. Zrepl snapshots the home dataset on a 15min cadence, replicating it to the backup dataset.
It also has a job to replicate to the backup server(s) at home when they’re reachable. Whenever it goes online at home, within 15mins it’s replicating to backup boxes … I’m sure you can set up something similar with sanoid/syncoid.
I keep meaning to setup a udev rule to trigger when it goes online at home, just haven’t gotten around to it yet.