PCI addresses with physaddr > 0xffffffff can't be mapped in
Enabling the BIOS option to enable decoding addresses above 4GB for PCIe devices causes ixgbe to fail attach due to:
2012-10-18T00:17:16.922070+00:00 headnode ixgbe: [ID 611667 kern.warning] WARNING: ixgbe0: Failed to map device registers 2012-10-18T00:17:16.922073+00:00 headnode ixgbe: [ID 611667 kern.warning] WARNING: ixgbe1: Failed to map device registers
The assigned address is indeed above 4GB. Disabling this option eliminates the problem.
Unfortunately the npe nexus driver does not support this. See the following comment in npe_bus_map:
648 case PCI_ADDR_MEM64: 649 /* 650 * We can't handle 64-bit devices that are mapped above 651 * 4G or that are larger than 4G. 652 */ 653 if (pci_rp->pci_phys_mid != 0 || pci_rp->pci_size_hi != 0) 654 return (DDI_FAILURE);
A bit more background. From digging into this, the x86 root nexus driver has never supported 64-bit addresses. In particular the nexus driver uses a struct regspec, which only allow for 32-bit quantities to be encoded. The interface between the root nexus driver and its parent driver are private. To deal with not needing to modify a bunch of other nexus drivers, we've used another flag that the PCI and NPE drivers use.
Updated by Electric Monk about 3 years ago
- Status changed from New to Closed
commit 1f0c5e612d8dfd7d1a8f4487cef753357b9909f5 Author: Robert Mustacchi <firstname.lastname@example.org> Date: 2017-12-04T17:23:22.000Z 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in 8703 pci/npe DDI_CTLOPS_REGSIZE should be 64-bit aware 8704 want OFF_MAX in the kernel 8705 unsupported 64-bit prefetch memory on pci-pci bridge Reviewed by: Patrick Mooney <email@example.com> Reviewed by: Jerry Jelinek <firstname.lastname@example.org> Reviewed by: Albert Lee <email@example.com> Approved by: Gordon Ross <firstname.lastname@example.org>