Bzfs_jobrunner - use a single shared jobconfig file to periodically create ZFS snapshots, replicate and prune, across source host and multiple destination hosts

OpenZFS
1

This v1.10.0 release contains some fixes and a lot of new features, including …

  • Improved compat with rsync,net
  • Added daemon support for periodic activities every N milliseconds, including for taking snapshots, replicating and pruning.
  • Added the bzfs_jobrunner companion program, which is a convenience wrapper around bzfs that simplifies periodically creating ZFS snapshots, replicating and pruning, across source host and multiple destination hosts, using a single shared jobconfig script.
  • Added --create-src-snapshots-* CLI options for efficiently creating periodic (and adhoc) atomic snapshots of datasets, including recursive snapshots.
  • Added --delete-dst-snapshots-except-plan CLI option to specify retention periods like sanoid, and prune snapshots accordingly.
  • Added --delete-dst-snapshots-except CLI flag to specify which snapshots to retain instead of which snapshots to delete.
  • Added --include-snapshot-plan CLI option to specify which periods to replicate.
  • Added --new-snapshot-filter-group CLI option, which starts a new snapshot filter group containing separate --{include|exclude}-snapshot-* filter options, which are UNIONized.
  • Added anytime and notime keywords to --include-snapshot-times-and-ranks.
  • Added all except keyword to --include-snapshot-times-and-ranks, as a more user-friendly filter syntax to say “include all snapshots except the oldest N (or latest N) snapshots”.
  • Log pv transfer stats even for tiny snapshots.
  • Perf: Delete bookmarks in parallel.
  • Perf: Use CPU cores more efficiently when creating snapshots (in parallel) and when deleting bookmarks (in parallel) and on --delete-empty-dst-datasets (in parallel)
  • Perf/latency: no need to set up a dedicated TCP connection if no parallel replication is possible.
  • For more clarity, renamed --force-hard to --force-destroy-dependents. --force-hard will continue to work as-is for now, in deprecated status, but the old name will be completely removed in a future release.
  • Use case-sensitive sort order instead of case-insensitive sort order throughout.
  • Use hostname without domain name within --exclude-dataset-property.
  • For better replication performance, changed the default of bzfs_no_force_convert_I_to_i form false to true.
  • Fixed “Too many arguments” error when deleting thousands of snapshots in the same ‘zfs destroy’ CLI invocation.
  • Make ‘zfs rollback’ work even if the previous ‘zfs receive -s’ was interrupted.
  • Skip partial or bad ‘pv’ log file lines when calculating stats.
1 Like
2

At this point I think an introduction to the project would serve most readers better than a changelog. So how about it? Tell us about the project and what makes it useful / cool / your favorite?

3

Hi Jim,

Good to talk to you.

bzfs is a lightweight, Python-based program designed to automate ZFS filesystem tasks like periodic and adhoc snapshot management, synchronization and backups. It’s built for sysadmins who need a clean, modular framework that’s both easy to extend and reliable in production, and fast. While tools like sanoid/syncoid offer robust ZFS management, bzfs emphasizes simplicity and customizability, making routine maintenance tasks both straightforward and adaptable. It’s my favorite because it combines Python’s clarity with practical automation, giving you a flexible foundation to tailor exactly to your needs.

1 Like