
This is what I've actually observed ever since I first started using this in FreeBSD 9.0 with the GEOM ELI 4k sector emulation. > My understanding is that a ZFS mirrored pool with two drives can read from both drives at the same time hence double the speed. The /testdb directory is a ZFS file system with recordsize=8k, chosen as ultimately it's intended to host a PostgreSQL database which uses an 8k page size. > Note that file /testdb/test is 16GB, twice the size of ram available in this system. > What is the exact dd command your running as that can have a huge impact on performance. > uffer from the original performance degradation I observed. I've yet to determine if this zpool setup as done by the bsdinstall script will However, instead of seeing the 950MB/s that I expected - which it what I see it with my alternative emulation - I'm seeing 450MB/s. As I wasn't making any headway in resolving this, I opted today to use the FreeBSD11.1 Beta 2 memstick image to create a basic FreeBSD 11.1 Beta 2 amd64 Auto(ZFS) installation to see if this would resolve the original issue I was having as I would be using ZFS-on-root and _auto_ashift=12 instead of my own emulation as described above. I'm seeing this behavior on FreeBSD 10.3R amd64 up through FreeBSD11.0 Stable.

However, after anywhere from 5 to 24 hours, performance would degrade down to less than 100MB/s for unknown reasons - server was essentially idle so it's a mystery to me why this occurs. > Using the above zpool configuration on this same hardware on FreeBSD 11-Stable, I was seeing read speeds of 950MB/s using dd (dd if=/testdb/test of=/dev/null bs=1m). > Then, the zpool mirrored pair would be something like the following:

> So, adaXp3 would show up as something like the following:

> adaXp3 - UFS with UUID labeled partition hosting a GEOM ELI layer using NULL encryption to emulate 4k sectors (done before ashift was an option) > I've been using FreeBSD+ZFS ever since FreeBSD 9.0, admittedly, with a different zpool layout which is essentially as follows: I've been posting to the freebsd-hackers list, but have moved to freebsd-fs list as this seemingly has something to do with FreeBSD+ZFS behavior and user Jov had previously cross-posted to this list for me:

> As I related in my post, my previous FreeBSD 11-Stable setup using this same hardware, I was seeing 950MB/s after bootup. > You don't explain why you believe there is degrading performance? As a user, this is a bit confusing to see it in /boot/nf but do a 'sysctl -a | grep min_auto_ashift' and see '_auto_ashift: 9' so felt it was worth mentioning.
EM0 NETMAP INSTALL
> The bsdinstall install script itself set _auto_shift=12 in /boot/nf yet, as you say, this doesn't do anything. > There's no need for it to be a tuneable as it only effects vdevs when they are created, which an only be done once the system is running. _auto_ashift is a sysctl only its not a tuneable, so setting it in /boot/nf won't have any effect.
