Feature #3316
open
Feature #2239: Integrate the virtio net driver
MSI(-X) support on legacy PCI.
0%
Description
The PCI 2.2 standard adds MSI support, and PCI 3.0 extended it to MSI-X.
Currently, illumos only supports MSI on PCI-express, but not on legacy PCI.
This feature is important to run illumos efficiently in kvm/qemu. Judging from the Linux numbers, MSI support should give at least a 2x performance improvement for virtio-net.
Unfortunately, I don't have time to work on this task.
Updated by Daniel Kimmel over 6 years ago
Based on Andriy's review here (https://www.illumos.org/rb/r/86/) it appears that MSI is supported for legacy PCI (he was able to get the virtio drivers working with MSI, at any rate).
Updated by Daniel Kimmel over 6 years ago
It turns out that Andriy wasn't using legacy PCI -- bhyve (the hypervisor he was testing virtio support for) uses a PCIe bus for virtio devices, while KVM (the hypervisor I tested on) uses PCI.
However, while investigating I ran across the function responsible for determining if MSI should be available: apic_check_msi_support(). For no documented reason that I could find, the only criteria it uses to decide whether MSI is available is whether any first-level children of the root device node are of type "pciex". It seems like this bug could be fixed simply by checking whether the first-level children are "pciex" or if they are for a device which supports PCI versions >= 2.2.
There is also a tunable apic_support_msi which you can set manually in /etc/system for a system that you know supports MSI; extremely basic testing on KVM (with Andriy's fix applied as well) indicates that this works perfectly fine for the virtio drivers.