I’m trying to understand the basics of how ZFS works, I’m not ready yet to think about optimizations for Databases etc.
I was testing large mp4 file transfer using sftp and nfs. As I undertand both sftp and nfs do synchronous writes?
When I transfer a file using sftp:
-
To the host data pool FreeBSD (recordsize 128K) it transfers in 14 seconds and zpool iostat shows activity for about 3-5 seconds after transfer.
-
To a zvol VM (volblocksize 1M) on data pool it transfers in 15 seconds but zpool iostat shows write activity for 10 more seconds.
-
To a zvol VM (volblocksize 128K) on data pool it transfers in 14 seconds but zpool iostat shows write activity for 3-5 more seconds.
-
Then I have a Windows VM on zroot which is a single SSD using img as disk and with recordsize of 128K. The transfer here takes 23 seconds but after it’s completed zpool iostat shows writes for 45 more seconds.
-
Tranfsfer to the host zroot also transfers in 14 seconds but zpool iostat shows there is about 40 seconds of writes after file transfer.
So the question is what is going on?
-
I don’t see any changes in the RAM on the host, it’s the same size as it was.
-
I didn’t realy look at the ARC, it’s 100%, I don’t know if it has meanig when here when doing writes?
-
I don’t have a SLOG.
Let’s say I download a file to the Windows VM and when the VM says completed I pull the power cord off the Host which is still doing those writes. Will the ISO file that was downloaded to the VM be corrupted? I will test this, just not today and maybe not on this Host 
Does ZFS keep the file I’m downloading in RAM and then puts it on the disk?
Or does it write the file to the LOG which is on the same Disk belonging to the pool and then rewrites it somewhere else on that disk and that’s why the 45 seconds worth of writes?
I don’t see any reads during those 45 seconds, so does that mean the file wasn’t put into LOG temporarily. I guess LOG wouldn’t be included into iostat reads.
LOG is never skipped when doing writes whether sync or async? I guess I could re read that in the ZFS Mastery book.
Yeah basically I’m trying to understand this in my own way because I’m only working with concepts and knowledge that I possess.
The file transfer speed tests might not be all that meaningful because my Router is a 1Gbps and so are the NICs from the Desktop to FreeBSD.
The link gets almost saturated at 108.7MB/s when transfering to the data pool and to the host zroot. It’s almost half that, 65.7MB/s, going to the windows VM on zroot SSD. I guess I see now that this might be a limitation of the windows drivers or the img disk format?
---------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
marabou 3.34T 291G 0 1.08K 0 426M
mirror-0 1.67T 144G 0 567 0 209M
da0 - - 0 274 0 104M
da1 - - 0 293 0 104M
mirror-1 1.67T 146G 0 541 0 217M
da2 - - 0 274 0 109M
da3 - - 0 266 0 109M
---------- ----- ----- ----- ----- ----- -----
zroot 44.9G 66.1G 0 0 0 0
ada0p2 44.9G 66.1G 0 0 0 0
---------- ----- ----- ----- ----- ----- -----
The HDDs belonging to the data pool are 7200 RPM (WDC WD2003FYYS-0)
The SSD for the zroot is KINGSTON SA400S37120G