Project

General

Profile

Feature #11376

PCIe speeds and feeds should be exposed to userland

Added by Robert Mustacchi 3 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Category:
kernel
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:

Description

As part of our broader efforts around phy management, we should make sure to go through and expose PCIe link widths and supported speeds. While this was initially added to the kernel in 11367, the version in 11367 did not expose this information to user land and also never updated itself after discovering the device. This data should be updated whenever the link changes such as through mechanisms added in 11373.

I tested this by booting these changes on a variety of different systems. I verified the following about different systems:

  • The slots and devices reported their corresponding capabilities correctly and that when there were differences between what was supported and what was present, that was accurately reflected in the topology information and in the kernel state.
  • Using the pcieb command to limit and therefore change the state of devices was accurately reflected in the kernel state and then in fmtopo.
  • Looked at prtconf and fmtopo information on a number of different systems with different PCIe devices to verify that they made sense.
  • Verified that sysevents were present (per 11373)
  • Verified that devices worked correctly as part of general regression testing for 11374 and 11375.

History

#1

Updated by Robert Mustacchi about 2 months ago

The following is an example of some of the information that is now exposed in topo:

hc://:product-id=X10SLM+-LN4F:server-id=haswell:chassis-id=0123456789/motherboard=0/hostbridge=0/pciexrc=0/pciexbus=1/pciexdev=0
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:product-id=X10SLM+-LN4F:server-id=haswell:chassis-id=0123456789/motherboard=0/hostbridge=0/pciexrc=0/pciexbus
=1/pciexdev=0
    label             string    MB
    FRU               fmri      hc://:product-id=X10SLM+-LN4F:server-id=haswell:chassis-id=0123456789/motherboard=0
    ASRU              fmri      dev:////pci@0,0/pci8086,c01@1
  group: authority                      version: 1   stability: Private/Private
    product-id        string    X10SLM+-LN4F
    chassis-id        string    0123456789
    server-id         string    haswell
  group: pci                            version: 1   stability: Private/Private
    link-maximum-width uint32    0x8
    link-current-width uint32    0x8
    link-maximum-speed uint64    0x1dcd65000
    link-current-speed uint64    0x1dcd65000
    link-supported-speeds uint64[]  [ 2500000000 5000000000 8000000000 ]

hc://:product-id=X10SLM+-LN4F:server-id=haswell:chassis-id=0123456789/motherboard=0/hostbridge=0/pciexrc=0/pciexbus=1/pciexdev=0/pciexfn=0
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:product-id=X10SLM+-LN4F:server-id=haswell:chassis-id=0123456789/motherboard=0/hostbridge=0/pciexrc=0/pciexbus
=1/pciexdev=0/pciexfn=0
    label             string    MB
    FRU               fmri      hc://:product-id=X10SLM+-LN4F:server-id=haswell:chassis-id=0123456789/motherboard=0
    ASRU              fmri      dev:////pci@0,0/pci8086,c01@1/pci1077,11@0
  group: authority                      version: 1   stability: Private/Private
    product-id        string    X10SLM+-LN4F
    chassis-id        string    0123456789
    server-id         string    haswell
  group: io                             version: 1   stability: Private/Private
    dev               string    /pci@0,0/pci8086,c01@1/pci1077,11@0
    driver            string    qede
    instance          uint32    0x0
    module            fmri      mod:///mod-name=qede/mod-id=250
  group: pci                            version: 1   stability: Private/Private
    device-id         string    8070
    extended-capabilities string    pciexdev
    class-code        string    20000
    vendor-id         string    1077
    assigned-addresses uint32[]  [ 3271622672 0 4153540608 0 131072 3271622680 0 4143972352 0 8388608 3271622688 0 4153737216 0 65536 ]
    vendor-name       string    QLogic Corp.
    device-name       string    FastLinQ QL41000 Series 10/25/40/50GbE Controller
    subsystem-name    string    FastLinQ QL41212HLCU 25GbE Adapter

Note, that the pci group for the device case is new. The function information didn't change, it's just here to show an example.

#2

Updated by Electric Monk about 2 months ago

  • Status changed from New to Closed

git commit b3d69c058376d802cdebbced341adcb3253b113c

commit  b3d69c058376d802cdebbced341adcb3253b113c
Author: Robert Mustacchi <rm@joyent.com>
Date:   2019-08-19T17:40:33.000Z

    11376 PCIe speeds and feeds should be exposed to userland
    11373 pcieb should enable link bandwidth notifications
    11374 Clean up pcieb CERRWARN and smatch
    11375 Clean up pcie module -Wno-uninitialized and -Wno-parentheses
    Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
    Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Reviewed by: Paul Winder <Paul.Winder@wdc.com>
    Approved by: Richard Lowe <richlowe@richlowe.net>

Also available in: Atom PDF