illumos gate

Upstreaming OS-8136 from SmartOS:

It uses the existing blockif delete functionality and essentially wires it up to the virtio block device.

To try to preserve the attribution of the work, his original change (w/ a few small tweaks) will be done as one PR, and a second PR will have the illumos-specific bits for blockif and the bhyve brand.

For testing, I ran the zfs TRIM tests in an Ubuntu VM. The one failure was due to

ERROR: dd if=/dev/urandom of=/var/tmp/testdir/largefile conv=notrunc,nocreat seek=1024 bs=1048576 count=3072 exited 1

Likely due to / being too small on the test VM to create the test file. All the other TRIM tests passed. Additionally, I ran dtrace -n 'pid$target::ioctl:entry /arg1==1074/ {}' -p PID_OF_BHYVE_PROCESS to verify DKIOCFREE ioctls were being issued during the zfs trim tests.

I then added nodelete as an option to one of the test disks, rebooted, and used lsblk -D to verify that it no longer showed discard support:

(Before adding nodelete)

ubuntu@eb2dc436-64e7-6fe3-8c9f-89c4c1053fc8:~$ lsblk -D
NAME    DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
vda          512      512B      16M         0
├─vda1         0      512B      16M         0
├─vda2         0      512B      16M         0
├─vda14        0      512B      16M         0
└─vda15        0      512B      16M         0
vdb          512      512B      16M         0
vdc          512      512B      16M         0
├─vdc1         0      512B      16M         0
└─vdc9         0      512B      16M         0
vdd          512      512B      16M         0
├─vdd1         0      512B      16M         0
└─vdd9         0      512B      16M         0
vde          512      512B      16M         0
├─vde1         0      512B      16M         0
└─vde9         0      512B      16M         0

After

ubuntu@eb2dc436-64e7-6fe3-8c9f-89c4c1053fc8:~$ lsblk -D
NAME    DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
vda          512      512B      16M         0
├─vda1         0      512B      16M         0
├─vda2         0      512B      16M         0
├─vda14        0      512B      16M         0
└─vda15        0      512B      16M         0
vdb          512      512B      16M         0
vdc          512      512B      16M         0
├─vdc1         0      512B      16M         0
└─vdc9         0      512B      16M         0
vdd          512      512B      16M         0
├─vdd1         0      512B      16M         0
└─vdd9         0      512B      16M         0
vde            0        0B       0B         0
├─vde1         0        0B       0B         0
└─vde9         0        0B       0B         0