I am trying to setup my TrueNAS Scale server to be something I can edit my photos off of. To help facilitate that, I put 1TB of RAM and made an nvme pool consisting of 2 mirrors with ashift 13. I also put in a intel x550-T2 dual port 10 gig nic in the server and my workstation, directly connected them and enabled SMB multichannel.
The first step in my photo workflow is using Photo Mechanic Plus (PM) to cull the photos. When I’ve opened the folder containing the photos in PM, it is very slow. Its like I’m working on a slow spinning drive and the app becomes very slow. It takes a long time to open the folder and gather the data it needs for sorting. It is at least a minute before I can start working. When I do start working, the app freezes up every so often while going through photos. It feels like there is lag and the server isn’t giving the data fast enough.
If I open the same folder locally on my workstation, its ready to go in about 5-10 seconds and have no issues.
so what am I missing? Why is the performance so slow?
Server Specs:
TrueNas Scale
X9DRH-7TF
dual Xeon E5-2630 v2
1TB RAM (16x Samsung 64GB PC3L-10600L - 8RX4 - LRDIMM Ram Memory (M386B8G70DE0)]
NVME Pool is 2 mirrors. NVME is connected to the PCIE slots on the motherboard with a jacobsparts m.2 to PCIe adapter
Mirror 0 - 2TB Intel 670p Series M.2 2280
Mirror 1 - 2TB Samsung 980 Pro
Desktop is Windows 10 with a 5800x3d on a B450 Aorus M with 64 GB of Mushkin Enhanced Redline Stiletto DDR4 3600
Do you have a second Windows machine with decent specs? If so, it might be worth doing a test by sharing a folder with lots of your photos from that machine, and accessing from the first. If the result looks pretty much identical to when you browse the TrueNAS share, you can conclude it’s a network issue.
OK, since we’ve eliminated both network and topology as bottlenecks for throughput, we’re left looking at a likely latency problem. SMB is an extremely chatty protocol, and it sounds like PhotoMechanic very likely initiates a ton of small requests that need to be fulfilled in order.
The next thing is going to be experimenting with different protocols. TrueNAS supports both NFS and iSCSI. NFS works essentially like SMB does, so you can try just enabling it on the NAS, setting it up on your laptop, and seeing if you get any improvement.
If that doesn’t help–or doesn’t help enough–you’re left with iSCSI, which is much higher performance, but also more of a pain to work with. iSCSI is essentially “a local disk” from your laptop’s perspective–it moves blocks down the wire, not files. But you can only access an iSCSI target from a single client at a time, and you can’t just “enable iSCSI” on your existing files, you’d need to create a new iSCSI volume, connect your laptop to it, format it NTFS, then begin copying files.
Photo Mechanic Tests - Time to open folder until I can start working
NFS - TN Server to Win Desktop Direct 10 Gig - Gave up after 12 minutes
SMB - TN Server to Win Desktop Direct 10 Gig - 3:50
SMB - Laptop to Win Desktop - 1 Gig via switch - 0:38
Local NVMe on Win Desktop - 0:22
@mercenary_sysadmin Would you be willing to test the program on your setup to rule out the NAS OS? They have a free full fledged trial, PM Plus is the version I use. Camera Bits is the developer. (I can only put two links in this post, so I saved them for the two below)
And there are a few links that have sample RAW files you can download:
@mercenary_sysadmin It looks like its a recordsize issue, which in hindsight makes total sense.
So where its slowing down is when PM is gathering sort data. Part of the sort data might be in those tiny xmp files. I tested out 16,32 and 64k recordsizes and all of them were under a minute. I’ll add 128,256 and 512 later and do multiple tests to get an average, but it looks like changing the recordsize is the fix.
So should I change recordsize away from 1M on my archive datasets as well? I won’t access most of them often.
The problem isn’t going to be in those tiny .xmp files; it sounds like Photo Mechanic is scanning at least the header area of each of the photos as well. The .xmp files will be stored in no larger a record than is necessary to contain them, regardless of recordsize–but if PM is trying to scan the first block of each of the actual photos, then you’ll have gone from pulling 1MiB per photo to only needing to pull 64K (or whatever) per photo.
You shouldn’t need to do that anywhere else. Recordsize only needs to be decreased when you have random access within large files, which obviously is the case for Photo Mechanic, but is not the case for the majority of your other data. Typically, you only really need to worry about it where you’ve got VMs or databases–which includes “flat-file databases” like SQLite.
