The Windows Server I have has it’s database files on a pair of SSDs in a mirror. The system itself is slow and SQL performance is bad since the SQL service is running on C:\
May I ask what your results were run on? Vdev setup, etc.
Most likely that stems from the 8k zvol. Would you agree?
I re-ran the tests to get the latency
Proxmox Server (vmhost01)
vmhost01:/zpool1# fio --name=randwrite --ioengine=posixaio --rw=randwrite --bs=64k --numjobs=8 --size=4g --iodepth=8 --runtime=60 --time_based --end_fsync=1 --group_reporting
randwrite: (g=0): rw=randwrite, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=posixaio, iodepth=8
...
fio-3.25
Starting 8 processes
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
Jobs: 7 (f=7): [F(5),_(1),F(2)][100.0%][eta 00m:00s]
randwrite: (groupid=0, jobs=8): err= 0: pid=3519160: Wed May 22 11:13:15 2024
write: IOPS=2210, BW=138MiB/s (145MB/s)(10.2GiB/75737msec); 0 zone resets
slat (nsec): min=1113, max=803710, avg=4502.69, stdev=4041.99
clat (usec): min=124, max=181643, avg=22942.51, stdev=10423.80
lat (usec): min=135, max=181649, avg=22947.01, stdev=10424.03
clat percentiles (usec):
| 1.00th=[ 693], 5.00th=[ 832], 10.00th=[ 1106], 20.00th=[ 19792],
| 30.00th=[ 21365], 40.00th=[ 22938], 50.00th=[ 24511], 60.00th=[ 26346],
| 70.00th=[ 27395], 80.00th=[ 29492], 90.00th=[ 32637], 95.00th=[ 35914],
| 99.00th=[ 45876], 99.50th=[ 47973], 99.90th=[ 68682], 99.95th=[ 86508],
| 99.99th=[158335]
bw ( KiB/s): min=83328, max=1814016, per=100.00%, avg=178639.07, stdev=22875.82, samples=959
iops : min= 1302, max=28344, avg=2790.72, stdev=357.44, samples=959
lat (usec) : 250=0.01%, 500=0.05%, 750=2.27%, 1000=6.90%
lat (msec) : 2=2.04%, 4=0.78%, 10=1.86%, 20=6.81%, 50=78.99%
lat (msec) : 100=0.25%, 250=0.04%
cpu : usr=0.65%, sys=0.24%, ctx=167996, majf=0, minf=764
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=99.9%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,167434,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
Run status group 0 (all jobs):
WRITE: bw=138MiB/s (145MB/s), 138MiB/s-138MiB/s (145MB/s-145MB/s), io=10.2GiB (10.0GB), run=75737-75737msec
Ubuntu KVM Server (vmhost02)
vmhost02:/zpool1$ sudo fio --name=randwrite --ioengine=posixaio --rw=randwrite --bs=64k --numjobs=8 --size=4g --iodepth=8 --runtime=60 --time_based --end_fsync=1 --group_reporting
randwrite: (g=0): rw=randwrite, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=posixaio, iodepth=8
...
fio-3.36
Starting 8 processes
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
randwrite: Laying out IO file (1 file / 4096MiB)
Jobs: 8 (f=8): [F(8)][100.0%][eta 00m:00s]
randwrite: (groupid=0, jobs=8): err= 0: pid=51968: Wed May 22 16:15:42 2024
write: IOPS=7525, BW=470MiB/s (493MB/s)(30.3GiB/66049msec); 0 zone resets
slat (nsec): min=810, max=12480k, avg=3605.62, stdev=18952.40
clat (usec): min=135, max=121471, avg=7718.77, stdev=3432.35
lat (usec): min=169, max=121475, avg=7722.38, stdev=3432.26
clat percentiles (usec):
| 1.00th=[ 586], 5.00th=[ 3523], 10.00th=[ 5342], 20.00th=[ 6194],
| 30.00th=[ 6587], 40.00th=[ 7046], 50.00th=[ 7373], 60.00th=[ 7767],
| 70.00th=[ 8225], 80.00th=[ 8848], 90.00th=[10945], 95.00th=[12911],
| 99.00th=[15795], 99.50th=[19530], 99.90th=[44303], 99.95th=[57934],
| 99.99th=[93848]
bw ( KiB/s): min=252288, max=2024960, per=100.00%, avg=532165.82, stdev=22605.73, samples=952
iops : min= 3942, max=31640, avg=8314.99, stdev=353.22, samples=952
lat (usec) : 250=0.01%, 500=0.44%, 750=1.46%, 1000=0.56%
lat (msec) : 2=1.13%, 4=2.00%, 10=81.24%, 20=12.70%, 50=0.39%
lat (msec) : 100=0.07%, 250=0.01%
cpu : usr=1.73%, sys=0.81%, ctx=505354, majf=0, minf=1780
IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=99.8%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,497075,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
Run status group 0 (all jobs):
WRITE: bw=470MiB/s (493MB/s), 470MiB/s-470MiB/s (493MB/s-493MB/s), io=30.3GiB (32.6GB), run=66049-66049msec