Feature #12792

bhyve upstream sync 2020 May

Added by Patrick Mooney 6 months ago. Updated 6 months ago.

Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


Sync upstream activity in FreeBSD's bhyve into illumos.
For now, we'll be ignoring the changes made to support save/restore.

Related issues

Related to illumos gate - Bug #12751: Update the cached MSI state when any MSI capability register is written.ResolvedMichael Zeller


Updated by Patrick Mooney 6 months ago

sjorge tested this atop SmartOS:

Applied this on top of SmartOS PI I build earlier today. Tested the following OS's and all still good
- FreeBSD
- NetBSD
- OpenBSD
- OmniOS
- Windows 10

FreeBSD and NetBSD was done on:
Supermicro X9DRi-LN4+/X9DR3-LN4+ + 2x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (Ivy Bridge EP, so It should have APICv)

FreeBSD, OpenBSD, OmniOS, and Windows:
Supermicro SSG-5029P-E1CTR12L + 1x Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz (Skylake EP (IIRC))


Updated by Michael Zeller 6 months ago

I applied this patchset on top of illumos-joyent and ran some tests.

Ran various linux guests such as Ubuntu (20.04/18.04), and Archlinux (kernel 5.6.15-arch1-1).

Physical Hardware:
Intel NUC (Version String: KYSKLi70.86A.0051.2018.0110.1800)
Intel(r) Core(tm) i7-6770HQ CPU @ 2.60GHz

Linux KVM with nested virt:
AMD Ryzen Threadripper 1950X 16-Core Processor

I also confirmed that I could run through the Ubuntu 20.04 installer multiple times without seeing the segfault umouse_request(), which is present without the resync. There's also an upstream patch that hardens this code even though the MSI patch included in the resync seems to fix the issue.

I verified that the bhyve virtual nvme device still works as there were a bunch of changes in that code. Finally I tested passing through a physical nvme drive into a guest.

[root@deepnest ~]# nvme list
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     NVME-0-5             bhyve-NVMe                               1           5.37  GB /   5.37  GB    512   B +  0 B   1.0
/dev/nvme1n1     BTPY72940FJ2256D     INTEL SSDPEKKF256G7L                     1         256.06  GB / 256.06  GB    512   B +  0 B   121P

[root@deepnest ~]# lsblk -f
NAME        FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
├─vda1      vfat   FAT32       2084-5FD4
└─vda2      ext4   1.0         8cdf7f8e-84a1-41af-b6be-3c7523d8eff7    3.5G    76% /
vdb         xfs                8b0387bd-299f-4823-9190-a4ce884674a3
nvme0n1     xfs                8b0387bd-299f-4823-9190-a4ce884674a3
└─nvme1n1p1 ext4   1.0         280d8aaf-b635-4c38-8c4b-46d20edc8e7e
[root@deepnest ~]# mount -t xfs /dev/nvme0n1 /mnt/a
[root@deepnest ~]# mount -t ext4 /dev/nvme1n1p1 /mnt/b
[root@deepnest ~]# cd /mnt/a
[root@deepnest a]# ls
foo  testfile
[root@deepnest a]# touch sync
[root@deepnest a]# ls
foo  sync  testfile
[root@deepnest a]# cd /mnt/b
[root@deepnest b]# ls
lost+found  test  testfile
[root@deepnest b]# touch sync
[root@deepnest b]# ls
lost+found  sync  test    testfile


Updated by Patrick Mooney 6 months ago

After a small fix, the mevent tests are happy again:

Executing test /opt/bhyvetest/tst/mevent/lists.delete.exe ... passed
Executing test /opt/bhyvetest/tst/mevent/read.disable.exe ... passed
Executing test /opt/bhyvetest/tst/mevent/read.pause.exe ... passed
Executing test /opt/bhyvetest/tst/mevent/read.requeue.exe ... passed


Tests passed: 4
Tests failed: 0
Tests ran:    4


Updated by Patrick Mooney 6 months ago

Both sjorge and Mike Zeller re-ran their tests after the mevent update and found them behaving properly as expected.


Updated by Electric Monk 6 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 154972aff898a787b38af3bab5b8d754b5a42447

commit  154972aff898a787b38af3bab5b8d754b5a42447
Author: Patrick Mooney <>
Date:   2020-06-12T15:11:53.000Z

    12792 bhyve upstream sync 2020 May
    Reviewed by: Mike Zeller <>
    Approved by: Robert Mustacchi <>


Updated by Patrick Mooney 6 months ago

  • Related to Bug #12751: Update the cached MSI state when any MSI capability register is written. added

Also available in: Atom PDF