Bug #8702

PCI addresses with physaddr > 0xffffffff can't be mapped in

Added by Robert Mustacchi 18 days ago.

Status:NewStart date:2017-10-04
Priority:NormalDue date:
Assignee:Robert Mustacchi% Done:

100%

Category:driver - device drivers
Target version:-
Difficulty:Medium Tags:

Description

Enabling the BIOS option to enable decoding addresses above 4GB for PCIe devices causes ixgbe to fail attach due to:

2012-10-18T00:17:16.922070+00:00 headnode ixgbe: [ID 611667 kern.warning] WARNING: ixgbe0: Failed to map device registers
2012-10-18T00:17:16.922073+00:00 headnode ixgbe: [ID 611667 kern.warning] WARNING: ixgbe1: Failed to map device registers

The assigned address is indeed above 4GB. Disabling this option eliminates the problem.

--

Unfortunately the npe nexus driver does not support this. See the following comment in npe_bus_map:

 648         case PCI_ADDR_MEM64:
 649                 /*
 650                  * We can't handle 64-bit devices that are mapped above
 651                  * 4G or that are larger than 4G.
 652                  */
 653                 if (pci_rp->pci_phys_mid != 0 || pci_rp->pci_size_hi != 0)
 654                         return (DDI_FAILURE);

A bit more background. From digging into this, the x86 root nexus driver has never supported 64-bit addresses. In particular the nexus driver uses a struct regspec, which only allow for 32-bit quantities to be encoded. The interface between the root nexus driver and its parent driver are private. To deal with not needing to modify a bunch of other nexus drivers, we've used another flag that the PCI and NPE drivers use.


Related issues

Related to illumos gate - Feature #8703: pci/npe DDI_CTLOPS_REGSIZE should be 64-bit aware New 2017-10-04
Related to illumos gate - Bug #8704: want OFF_MAX in the kernel New 2017-10-04
Related to illumos gate - Bug #8705: unsupported 64-bit prefetch memory on pci-pci bridge New 2017-10-04

History

#1 Updated by Robert Mustacchi 18 days ago

  • Related to Feature #8703: pci/npe DDI_CTLOPS_REGSIZE should be 64-bit aware added

#2 Updated by Robert Mustacchi 18 days ago

  • Related to Bug #8704: want OFF_MAX in the kernel added

#3 Updated by Robert Mustacchi 18 days ago

  • Related to Bug #8705: unsupported 64-bit prefetch memory on pci-pci bridge added

Also available in: Atom