I have 2 sets of NVMes I want to put in my server, two PCIE4 Samsungs and two PCIE 3 Intels. If I want the lowest latency reads, am I better off making a 4 way mirror or should I make a pool of 2 mirrored ssds?
Two mirror vdevs will outperform a single four-way mirror vdev on most workloads, even read-specific workloads.
The reason is that you’ll hit both vdevs with a single-process read, whereas you usually will not hit more than one disk in a mirror with a single-process read. Generally speaking, you need multiple read processes to distribute read workload near-evenly across both sides of a mirror vdev.
If I don’t care about the pools data and only care about performance, should I do 4 vdevs?
Correct. The highest performing topology for any given number of drives is a pool of single drive vdevs.
Just remember, lose ANY drive for ANY reason, and you’ll lose the entire pool of singletons along with it.