Zpool.cache corrupted?

OpenZFS
1

I have 3 machines on which I’m using ZFS, my favorite file system, under both Linux and FreeBSD.

One of them is used as a file server (over NFS), backup and media server. The last couple of days it started behaving badly:

When booting the machine the systemd service that imports zfs pools fails with a message on the line of “cannot import pool zzz”. I’m not using the zfs.service provided by the distribution which imports pools using cache, but a custom one in which each pool is imported with “zpool import -f pool”. This way worked perfectly for years.

If I delete /etc/zfs/zpool.cache and reboot the machine, everything gets back to normal.

This does not happen in FreeBSD, where I also import pools using zpool import -f.

All pools have the property “cachefile -” (null, default).

Two questions:

  1. Why is this happening lately and only in Linux?
  2. I think that /etc/zfs/zpool.cache gets corrupted somehow. Is there a way to avoid creating that file that seems to give so much trouble lately?
2

Why are you running -f(orce) on every import?

3

As I’m using both Linux and FreeBSD I noticed that without -f the pools cannot be imported in either system when alternately booting them.

4

Actually I think I was wrong: in FreeBSD I imported the pools once, the first time, and from then on the system is just importing the pools when it boots, most probably by using zpool.cache.

Only in Linux I use zpool import -f, because otherwise the imports fail when booting after FreeBSD.

5

It would be better to add a step during shutdown to export the pool, rather than blindly forcing on every boot, IMO. Which would also leave us with less questions about why your setup is having problems; “I’ve gotten rid of the distro’s service unit and written my own” is kind of impossible to get past, particularly without actually showing us the exact service unit you are using.

I apologize if this comes off as cranky; I don’t intend it to be. But when you introduce something custom like this, you do have to trot it out for inspection when asking for help–nobody else has ever seen it, so we can’t make any assumptions about it, you know? :cowboy_hat_face:

6

More directly to your point, yes there is a way to disable zpool.cache; you zpool set cachefile=none poolname. After which you need to make certain that you’ve systemctl disable’d zfs-import-cache.service, and (normally) enable’d zfs-import-scan.service.

But since you’re well off the ranch, I’m not entirely certain this will fix your problem as it currently sits, nor at which points enabling a distro provided service will clash with your custom written service, so YMMV.

7

Thanks for your input and no worries and I’ll also give you some detailed answers:

1 - The service I wrote is very straightforward:

zfs.service

[Unit]
Description=Import ZFS pools
After=systemd-remount-fs.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecCondition=/bin/grep -q "^zfs " /proc/modules
ExecStart=/sbin/zpool import -f pool1
ExecStart=/sbin/zpool import -f pool2
...
ExecStart=/sbin/zpool import -f poolz
TimeoutSec=23

[Install]
WantedBy=zfs-import.target

2 - Only this custom zfs.service is enabled. All distribution provided zfs services are disabled:

zfs-import-cache.service
zfs-import-scan.service
zfs-import.service
zfs-load-key.service
zfs.mount.service
zfs-mount@.service
zfs-scrub@.service
zfs-trim@.service
zfs-volume-wait.service
zfs-zed.service

If I try to use them as provided, they fail as #3 below.

3 - On a Linux boot right after a FreeBSD boot, without -f the pool imports fail with:

zpool: cannot import 'pool1': pool was previously in use from another system
zpool: Last accessed by host1 (hostid=8645113a) at Wed Jul 15 09:09:44 2026
zpool: The pool can be imported, use import -f to import the pool.

4 - FreeBSD never complains and happily imports the pools after a previous Linux boot.

5 - There are other things about ZFS behaving a bit differently under Linux and FreeBSD, but I’ll bring them up as needed.

8

Sounds like the simplest solution is an export command inserted into the shutdown sequence on the FreeBSD side, followed by switching back to the normal services. That will stop Linux complaining that another system currently has dibs on the pool, and get you back into a more recognizable state as a bonus.

Ideally, you really should do that for both sides. I think you’re correct that FreeBSD is using its own zpool.cache to import your pool(s), because in theory it should also be complaining about the Linux side still having dibs on the pool(s) when it boots.

9

Isn’t a zpool export kind of too much?

One of the pools holds my /home and I was worried to export that pool on shutdown of a running system. Also there are a few ‘-rbind’ mounts on Linux or ‘nullfs’ mounts on FreeBSD that would be broken on export.

I remember there was a while ago an option to not check a previous use by another system, something related to the machine id or host id, but the ZFS developers eliminated it at some point a few years ago and I was forced to use that zpool import -f hack.

10

What follows is a bit of ranting, so please skip it if you don’t like that kind of stuff.

Ok, so… developers decided that it’s more important to let me know that the pool was used in another system but, you know, well… you can still import it with -f if you really want to.

First, what could have possibly happened during the use of the pool on another OS on the same machine that would warrant warning me in such a way instead of simply letting me use the pool?

Second, all this export-import thing that is recommended is such an annoying way yo do things. Even if the pools are imported with zpool.cache the pools log is full of such import actions. All this could be easily avoided.

In the end the pool is a partition (or partitions) on disks, just sitting there, waiting to be used. They are no different than a BTRFS partition that just sits there waiting for its volumes and sub-volumes to be mounted, just the way ZFS datasets are.

I know that the FreeBSD boot loader is scanning disks to check if there are any ZFS pools and then if any of them has a bootfs dataset to boot the OS. Probably the Linux ZFS module does the same thing when loaded from initramfs, but I’m not sure, I never tried that.

Ok, so you scan the disks, you find the pools and then you force me to ‘import’ them to be able to use them. Why not instead use the sensible way of somehow ‘open’-ing them and make them ready to have the datasets mounted using the proper fstab entries?

But no, the developers, in their infinite wisdom, decided that they can’t do things the standard way, they must use their own, different and annoying way… that will show’em, we are not lik’em!

Fine, zpool import works, but why can’t you make it work WITHOUT being annoying?

ZFS is such outstanding filesystem, but it has these little quirks here and there that, sometimes, makes it so frustrating in use.