Project

General

Profile

Feature #12737

sync shadow PCIR_COMMAND with real one for bhyve pass-thru

Added by Patrick Mooney 10 months ago. Updated 9 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
cmd - userland programs
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
bhyve
Gerrit CR:

Description

Upstreaming OS-8143 from SmartOS:

This is a follow up to OS-8117. This pulls in an initially skipped userland diff for pci passthru.
The diff comes from FreeBSD commit 8e4b03df46ab (r348779).

Additionally we want to grab https://github.com/freebsd/freebsd/commit/bdad744823ebf970815d691c9ec71236f6f0d90b ahead of the next re-sync as the commit is directly related to r348779.

To test this change I booted a debug platform with the patch on an intel NUC.

I setup loader.conf.local to look like:

[root@hollownest ~]# cat /opt/local/bin/mount-usb^C
[root@hollownest ~]# cat /usb
usbkey/   usbstick/
[root@hollownest ~]# cat /usbstick/boot/loader.conf.local
ppt_aliases_load=YES
ppt_aliases_type=file
ppt_aliases_name=/overlay/etc/ppt_aliases
ppt_aliases_flags="name=/etc/ppt_aliases" 

ppt_matches_load=YES
ppt_matches_type=file
ppt_matches_name=/overlay/etc/ppt_matches
ppt_matches_flags="name=/etc/ppt_matches" 

With each file looking like:

[root@hollownest ~]# cat /usbstick/overlay/etc/ppt_matches
pci8086,390a
[root@hollownest ~]# cat /usbstick/overlay/etc/ppt_aliases
ppt "/pci@0,0/pci8086,a118@1d/pci8086,390a@0" 

This sets aside the nvme device in the NUC for use with a bhyve VM.

[root@hollownest ~]# pptadm list -a -o dev,vendor,device,path
DEV        VENDOR DEVICE PATH
/dev/ppt0  8086   f1a5   /pci@0,0/pci8086,a118@1d/pci8086,390a@0

Using the above device I updated the VM with the following json payload:

{
  "pci_devices": [
    {
      "path": "/devices/pci@0,0/pci8086,a118@1d/pci8086,390a@0",
      "pci_slot": "0:8:0" 
    }
  ]
}

Inside the VM I provisioned an ext4 filesystem and wrote some data to a file.

[root@deepnest ~]# lsblk /dev/nvme1n1 -f
NAME FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
nvme1n1
│
└─nvme1n1p1
     ext4   1.0         280d8aaf-b635-4c38-8c4b-46d20edc8e7e  221.8G     0% /ppt
[root@deepnest ~]# echo hello > /ppt/testfile
[root@deepnest ~]# cat /ppt/testfile
hello

Additionally I had sjorge in the community test the patch on his hardware. He was able to passthrough the previously problematic NIC with multiple BARs to his Windows VM and everything worked as expected.

#1

Updated by Patrick Mooney 9 months ago

  • Status changed from New to In Progress
#2

Updated by Electric Monk 9 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit e43213726a6068355df89cb553a223328af375f4

commit  e43213726a6068355df89cb553a223328af375f4
Author: Michael Zeller <mike@mikezeller.net>
Date:   2020-05-23T16:38:23.000Z

    12737 sync shadow PCIR_COMMAND with real one for bhyve pass-thru
    Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
    Reviewed by: Jason King <jason.king@joyent.com>
    Reviewed by: Jorge Schrauwen <jorge@blackdot.be>
    Approved by: Robert Mustacchi <rm@fingolfin.org>

Also available in: Atom PDF