Project

General

Profile

Actions

Feature #14895

closed

System should report PCIe 32.0 and 64.0 GT/s speeds

Added by Robert Mustacchi about 2 months ago. Updated about 1 month ago.

Status:
Closed
Priority:
Normal
Category:
kernel
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
External Bug:

Description

There are several properties that are used in the devinfo tree to relate information about PCIe speeds and feeds to userland and into tools such as piceadm, fmtopo, etc. We should go through and update these to report information about PCIe Gen 5 32 GT/s operation and while we're there, PCIe Gen 6.0 64 GT/s operation. While Gen 6 devices aren't accessible now, this'll still be useful. This covers updates to:

  • The core pcie driver
  • pcieadm
  • pcieb

Note, fmtopo doesn't need an explicit update since it just takes everything directly from the devinfo properties.

Actions #1

Updated by Electric Monk about 2 months ago

  • Gerrit CR set to 2296
Actions #2

Updated by Robert Mustacchi about 2 months ago

To test this, I've been going through and looking at several things. First, with this now in place, we get the correct speed for several bridges and devices that we weren't previously:

rm@gilgamesh ~ $ /usr/lib/pci/pcieadm show-devs pcieb
BDF     TYPE           DRIVER         DEVICE
0/5/1   PCIe Gen 4x4   pcieb0         Unknown device: 0x14aa
0/7/1   PCIe Gen 5x16  pcieb1         Unknown device: 0x14a7
40/7/1  PCIe Gen 5x16  pcieb2         Unknown device: 0x14a7
80/1/1  PCIe Gen 1x16  pcieb3         Unknown device: 0x14ab
80/7/1  PCIe Gen 5x16  pcieb4         Unknown device: 0x14a7
c0/5/2  PCIe Gen 2x1   pcieb5         Unknown device: 0x14aa
c1/0/0  PCIe Gen 2x1   pcieb9         AST1150 PCI-to-PCI Bridge
c0/5/3  PCIe Gen 2x1   pcieb6         Unknown device: 0x14aa
c0/7/1  PCIe Gen 5x16  pcieb7         Unknown device: 0x14a7
c0/7/2  PCIe Gen 5x16  pcieb8         Unknown device: 0x14a7
rm@gilgamesh ~ $ set -o vi
rm@gilgamesh ~ $ /usr/lib/pci/pcieadm show-devs -s pcieb
BDF     DRIVER         MAXSPEED  CURSPEED  MAXWIDTH  CURWIDTH  SUPSPEEDS
0/5/1   pcieb0         16.0 GT/s 16.0 GT/s x4        x4        2.5,5.0,8.0,16.0
0/7/1   pcieb1         32.0 GT/s 32.0 GT/s x16       x16       2.5,5.0,8.0,16.0,32.0
40/7/1  pcieb2         32.0 GT/s 32.0 GT/s x16       x16       2.5,5.0,8.0,16.0,32.0
80/1/1  pcieb3         32.0 GT/s 2.5 GT/s  x16       x16       2.5,5.0,8.0,16.0,32.0
80/7/1  pcieb4         32.0 GT/s 32.0 GT/s x16       x16       2.5,5.0,8.0,16.0,32.0
c0/5/2  pcieb5         5.0 GT/s  5.0 GT/s  x1        x1        2.5,5.0
c1/0/0  pcieb9         5.0 GT/s  5.0 GT/s  x1        x1        2.5,5.0
c0/5/3  pcieb6         5.0 GT/s  5.0 GT/s  x1        x1        2.5,5.0
c0/7/1  pcieb7         32.0 GT/s 32.0 GT/s x16       x16       2.5,5.0,8.0,16.0,32.0
c0/7/2  pcieb8         32.0 GT/s 32.0 GT/s x16       x16       2.5,5.0,8.0,16.0,32.0

Next, we can look at the corresponding data in fmtopo:

hc://:product-id=ONYX:server-id=gilgamesh:chassis-id=To-be-filled-by-O.E.M./motherboard=0/
hostbridge=1/pciexrc=1/pciexbus=2/pciexdev=0
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:product-id=ONYX:server-id=gilgamesh:chassis-id=To-be
-filled-by-O.E.M./motherboard=0/hostbridge=1/pciexrc=1/pciexbus=2/pciexdev=0
    label             string    MB
    FRU               fmri      hc://:product-id=ONYX:server-id=gilgamesh:chassis-id=To-be
-filled-by-O.E.M./motherboard=0
    ASRU              fmri      dev:////pci@0,0/pci1022,14a7@7,1
  group: authority                      version: 1   stability: Private/Private
    product-id        string    ONYX
    chassis-id        string    To-be-filled-by-O.E.M.
    server-id         string    gilgamesh
  group: pci                            version: 1   stability: Private/Private
    link-maximum-width uint32    0x10
    link-current-width uint32    0x10
    link-maximum-speed uint64    0x773594000
    link-current-speed uint64    0x773594000
    link-supported-speeds uint64[]  [ 2500000000 5000000000 8000000000 16000000000 32000000000 ]

Let's now go prove that the pcieb utility works. Here's an example of us taking the following part of the three and converting it to operate at PCIe Gen 1 speeds:

rm@gilgamesh ~ $ /usr/lib/pci/pcieadm show-devs c0/5/2 c1/0/0 c2/0/0
c0/5/2  PCIe Gen 2x1   pcieb5         Unknown device: 0x14aa
c1/0/0  PCIe Gen 2x1   pcieb9         AST1150 PCI-to-PCI Bridge
c2/0/0  PCI            vgatext0       ASPEED Graphics Family
rm@gilgamesh ~ $ pfexec /usr/lib/pci/pcieb /devices/pci@bb,0/pci1022,14aa@5,2:devctl
Bridge target speed: 5.0 GT/s (gen2)
rm@gilgamesh ~ $ pfexec /usr/lib/pci/pcieb -s gen1 /devices/pci@bb,0/pci1022,14aa@5,2:devctl
rm@gilgamesh ~ $ pfexec /usr/lib/pci/pcieb /devices/pci@bb,0/pci1022,14aa@5,2:devctl
Bridge target speed: 2.5 GT/s (gen1)
Flags: Admin Set Speed
rm@gilgamesh ~ $ pfexec /usr/lib/pci/pcieb -x /devices/pci@bb,0/pci1022,14aa@5,2:devctl
rm@gilgamesh ~ $ /usr/lib/pci/pcieadm show-devs -s c0/5/2 c1/0/0 c2/0/0
BDF     DRIVER         MAXSPEED  CURSPEED  MAXWIDTH  CURWIDTH  SUPSPEEDS
c0/5/2  pcieb5         5.0 GT/s  2.5 GT/s  x1        x1        2.5,5.0
c1/0/0  pcieb9         5.0 GT/s  2.5 GT/s  x1        x1        2.5,5.0
c2/0/0  vgatext0       --        --        --        --        --
rm@gilgamesh ~ $ /usr/lib/pci/pcieadm show-devs c0/5/2 c1/0/0 c2/0/0
BDF     TYPE           DRIVER         DEVICE
c0/5/2  PCIe Gen 1x1   pcieb5         Unknown device: 0x14aa
c1/0/0  PCIe Gen 1x1   pcieb9         AST1150 PCI-to-PCI Bridge
c2/0/0  PCI            vgatext0       ASPEED Graphics Family
rm@gilgamesh ~ $ pfexec /usr/lib/pci/pcieb /devices//pci@0,0/pci1022,14a7@7,1:devctl
Bridge target speed: 32.0 GT/s (gen5)
Actions #3

Updated by Electric Monk about 1 month ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 8942719269eba26304a9de8478e842928c665dfc

commit  8942719269eba26304a9de8478e842928c665dfc
Author: Robert Mustacchi <rm@fingolfin.org>
Date:   2022-08-17T18:16:41.000Z

    14895 System should report PCIe 32.0 and 64.0 GT/s speeds
    14896 pcieb speed overrides should check link cap
    Reviewed by: Rich Lowe <richlowe@richlowe.net>
    Reviewed by: Andy Fiddaman <illumos@fiddaman.net>
    Approved by: Patrick Mooney <pmooney@pfmooney.com>

Actions

Also available in: Atom PDF