Project

General

Profile

Feature #11612

x86 PCI enumeration should not rely on bios max bus

Added by Robert Mustacchi about 1 year ago. Updated 7 months ago.

Status:
Closed
Priority:
Normal
Category:
kernel
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

Traditionally, on an x86 BIOS based system, we rely on the PCI BIOS specification which has a series of BIOS calls one can make to determine information about the PCI devices in the system. One of these values is supposed to contain the maximum PCI bus number that should be scanned for devices.

Unfortunately, we've found many cases where this information is wrong. In particular, I discovered this is the case for all of the SMCI Richmond, Tenderloin, and Hallasan systems. On these systems we were only seeing one socket's worth of built-in PCI devices from the 'uncore'. This made me very suspicious. Many other systems have the second socket's PCI bus as 0xff. However, the value of pci_bios_maxbus was much lower! Using kmdb, I confirmed my suspicion:

[3]> pci_bios_maxbus/D
pci_bios_maxbus:
pci_bios_maxbus:134             
[3]> 0::rdpcicfg 0xff 0t19 0
6fa88086

So clearly, the BIOS is lying here. We actually have a device that's above the max bus. While we should probably still trust it to determine what mechanism we're using. We're going to instead just treat the max bus like the EFI case and just scan all of them.

This has been tested on a number of different platforms in conjunction with 11609. These include:

  • Sandy Bridge 2s
  • Ivy Bridge 2s
  • Haswell 1s/2s
  • Broadwell 2s
  • Skylake 2s
  • Cascade Lake 2s
  • Kaby Lake NUC
  • Coffee Lake Desktop system
  • AMD EPYC system

Related issues

Related to illumos gate - Bug #11861: hostbridge topo module should be hardened to handle empty bussesClosedRobert Mustacchi

Actions
Related to illumos gate - Bug #11860: Use _CRS instead of _BBN to get assigned PCI busClosed

Actions
#1

Updated by Robert Mustacchi about 1 year ago

  • Related to Bug #11861: hostbridge topo module should be hardened to handle empty busses added
#2

Updated by Robert Mustacchi about 1 year ago

This hasn't gone back yet as a few folks discovered that on particular system utilities linked against libtopo were crashing due to a combination of 11861 and 11860.

#3

Updated by Robert Mustacchi about 1 year ago

  • Related to Bug #11860: Use _CRS instead of _BBN to get assigned PCI bus added
#4

Updated by Electric Monk 7 months ago

  • Status changed from New to Closed

git commit eb00b1c8a31c2253a353644606388dff5b0e0275

commit  eb00b1c8a31c2253a353644606388dff5b0e0275
Author: Robert Mustacchi <rm@joyent.com>
Date:   2020-03-24T22:27:39.000Z

    11609 Want modern Intel IMC driver
    11612 x86 PCI enumeration should not rely on bios max bus
    Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Reviewed by: Rob Johnston <rob.johnston@joyent.com>
    Approved by: Gordon Ross <gordon.w.ross@gmail.com>

Also available in: Atom PDF