Dataset is reported as empty, but imo it contains all my data

OpenZFS
1

Hi all,

I am new to ZFS, I created a pool across 2 discs:

sudo zpool create -o ashift=12 -O xattr=sa -O compression=lz4 -O atime=off zpool0 mirror /dev/sdb /dev/sdc -f

Then I created a dataset on the pool:

sudo zfs create zpool0/data1
sudo zfs set canmount=on zpool0/data1
sudo zfs set mountpoint=/zpool0/data1 zpool0/data1

I automount the dataset with this piece of nix configuration:

fileSystems."/zpool0/data1" = {
  device = "zpool0/data1";
  fsType = "zfs";
  # the zfsutil option is needed when mounting zfs datasets without "legacy" mountpoints
  options = [ "zfsutil" ];

};

I put all my pictures in /zpool0/data1

But now zfs list tells me:

$ zfs list
NAME           USED  AVAIL  REFER  MOUNTPOINT
zpool0         771G  2.76T   771G  /zpool0
zpool0/data1    96K  2.76T    96K  /zpool0/data1

Why does it say there is only 96K in /zpool0/data1?

Some more commands I checked:

$ du -h --max-depth=1 /zpool0/
772G	/zpool0/data1
772G	/zpool0/

$ mount | grep zpool0
zpool0/data1 on /zpool0/data1 type zfs (rw,noatime,xattr,noacl,casesensitive)
zpool0 on /zpool0 type zfs (rw,noatime,xattr,noacl,casesensitive)

I noted all my commands down, I may have made mistakes in the noting down.

My sources:
Sorry can only put 2 links in post

2

zfs list -rt all zpool0, please.

3 
$ zfs list -rt all zpool0
NAME                      USED  AVAIL  REFER  MOUNTPOINT
zpool0                    771G  2.76T   771G  /zpool0
zpool0/data1               96K  2.76T    96K  /zpool0/data1
zpool0/data1@2024-12-27     0B      -    96K  -

I made that snapshot just this afternoon with sudo zfs snapshot zpool0/data1@2024-12-27

4

Ok, so there’s definitely no data in that dataset. This leaves the possibility of data inside the root dataset, camouflaged by the presence of an empty dataset on top of it, or data in your root filesystem camouflaged by the pool being mounted on top of it.

zfs rename zpool0/data1 zpool0/test ; ls -l /zpool0/data1

If you don’t see any files in a directory named data1 after renaming the dataset data1, then:

zpool export zpool0 ; ls -l /zpool0/data1

And see if it’s there.

edit: to be clear, we are seeing 771G on zpool0, in its root dataset. You’ve phrased things as though this is a newly created pool, but haven’t been explicit on that score. Is that 771G your missing pictures, or is that other data that you already knew about on the root dataset?

5

Yeah, it’s strange, maybe it’s my mounting shenanigans, that zfs set part? Or perhaps my notes are wrong and I did something different than I noted down. I’ve been using this data(set) for months without issue, but just recently noticed the weirdness in zfs list -rt all zpool0. I tried your command:

$ sudo zfs rename zpool0/data1 zpool0/test
[sudo] password for freek: 
cannot unmount '/zpool0/data1': unmount failed

Should I unmount first or does this already say something?

If I look in the folders, everything appears to be in /zpool0/data1:

[freek@trantor:~]$ ls /zpool0/
data1

[freek@trantor:~]$ ls /zpool0/data1/
Pictures

[freek@trantor:~]$ ls /zpool0/data1/Pictures/ | head
1982-2003
2004
2005
2006
2007
2008
2009
2010
2011
2012

This is what I expect, months ago I copied my Pictures folder (771G) into /zpool0/data1.

Ok, I’ll just unmount and rename, I don’t really onderstand what is happening. Did I make folder by hand that shouldn’t have been made by hand back then?

$ sudo zfs rename zpool0/data1 zpool0/test
[sudo] password for freek: 
cannot unmount '/zpool0/data1': unmount failed

[freek@trantor:~]$ umount /zpool0/data1 
umount: /zpool0/data1: must be superuser to unmount.

[freek@trantor:~]$ sudo umount /zpool0/data1 
umount: /zpool0/data1: not mounted.

[freek@trantor:~]$ sudo umount /zpool0

[freek@trantor:~]$ sudo zfs rename zpool0/data1 zpool0/test

[freek@trantor:~]$ sudo mount /zpool0
mount: /zpool0: can't find in /etc/fstab.

[freek@trantor:~]$ sudo mount /zpool0/data1/
filesystem 'zpool0/data1' cannot be mounted, unable to open the dataset

[freek@trantor:~]$ sudo mount /zpool0/test
mount: /zpool0/test: can't find in /etc/fstab.

[freek@trantor:~]$ sudo mount /zpool0
mount: /zpool0: can't find in /etc/fstab.

[freek@trantor:~]$ ll /zpool0/
total 1
drwxr-xr-x 2 freek users 2 25 jun  2024 data1

[freek@trantor:~]$ ll /zpool0/data1/
total 0

[freek@trantor:~]$ zpool export zpool0
cannot unmount '/zpool0/data1': permission denied

[freek@trantor:~]$ sudo zpool export zpool0

[freek@trantor:~]$ ls -l /zpool0/data1
total 0

[freek@trantor:~]$ ls -l /zpool0/
total 4
drwxr-xr-x 2 root root 4096 27 dec 21:33 data1

Did I mount zpool0 at /zpool0/data1 or something?

6

It’s possible. I’m not familiar with the style of configuration you were using to mount things; it might very well have ignored the data1 part and mounted the ROOT dataset of zpool0 at /zpool0/data1.

You don’t normally screw around with datasets in /etc/fstab or similar at all; typically zfs handles all of its own mounting directly.

7

Ah, I didn’t know that just setting it all up would have made it auto-mount, perhaps that was my classical thinking, I always prefer to use the tools and go for minimal.

I’ll try to redo it without the automounting… As soon as I get this headless server to join my tailnet again so I can see what’s going on (I guess it’s hanging somewhere during boot because of the dataset renaming? Not what I would have expected btw, I thought after all this that I had simply created a dataset (data1) but then somehow mounted the zpool directly to some folder named data1 on /zpool0 (which would mean the snapshot is useless as well), ignoring the dataset itself. So I don’t understand how renaming the unused dataset causes a hang during boot…

A well, lessons have been learned, thanx a lot for the help!

1 Like
8

Ok, had to drag the old Fractal Design Define R2 out from behind the chistmas tree:

IMG_4210.HEIC
IMG_4210.HEIC3024×4032 3.43 MB

I played around a bit. When I changed the dataset name I got failed to mount /zpool0/data1 and the machine hang with You are in emergency mode ... ... Give root password for maintenance or press control-D to continue

Some more info:
This is the line in fstab:
zpool/data0 /zpool/data0 zfs zfsutil 0 0

When I removed the relevant line from fstab (bij removing that nix expression), there was simply nothing in /zpool0/data1.

When I rename data1 to test, there would be no data visible but zfs reported the name change:

sorry can add only 1 image
But nothing would auto-mount.

I guess I’ll just start over, make the pool again, make the dataset again, don’t do anything with fstab. I shouldn’t have done that, now that I re-read the nixos wiki (ZFS - NixOS Wiki). I didn’t use legacy mount points.

Again, thanx for the help, enjoy the rest of the holiday!

9

You too, friend! Sorry your initial experience was a bit annoying. ZFS, I’m pretty expert on. NixOS… I’m not even a proper noob on THAT. :sweat_smile:

10

Just to close this off properly,

I created a new dataset:

sudo zfs create zpool0/data2

rsynced all data from /zpool0/data1 (which is not a dataset but just seems to be a mount point).

After that it looks as expected:

[freek@trantor:/zpool0]$ zfs list -rt all zpool0
NAME                      USED  AVAIL  REFER  MOUNTPOINT
zpool0                   1.51T  2.01T   771G  /zpool0
zpool0/data1               96K  2.01T    96K  /zpool0/data1
zpool0/data1@2024-12-27     0B      -    96K  -
zpool0/data2              771G  2.01T   771G  /zpool0/data2

I didn’t need to do anything with auto-mounting, ‘canmount’ settings or fastab, the dataset is just mounted as I want after a reboot.

Now I’ll just remove the data from zpool1 , remove the data1 dataset, rename data2 to data1 and all is as intended after all.

Again thanx for the help. :vulcan_salute:

Edit: and indeed, all is well now:

[freek@trantor:~]$ zfs list -rt all zpool0
NAME           USED  AVAIL  REFER  MOUNTPOINT
zpool0         771G  2.76T    96K  /zpool0
zpool0/data1   771G  2.76T   771G  /zpool0/data1
2 Likes