Syncoid very slow when transferring small datasets

OpenZFS
1

Hello,

I use syncoid to transfer data between two servers on a gigabit LAN. The transfer speed is more than 100 MiB/s with larger datasets. However, with very small datasets (df says <20K) , the speed is very slow ~ 7KiB/s. Is there any way to speed the data-transfer for these small datasets?

Thanks

2

What you’re seeing isn’t actually “data transfer speed” it’s the speed of your pool in handling small metadata operations. I’m guessing your pool is rust.

3

Yes, both end are rust drives. Should i just flatten out all these datasets in a single dataset? The datasets are docker compose and volume files.

4

I can’t say if that’s what you “should” do, but it’s the only way you’re going to see replication speed up, short of moving to SSDs.

Replicating a ton of very small datasets instead of one large one is slow for the same reason copying 1GiB of 4KiB individual files goes (much, much) slower than copying a single 1GiB file would.