Project

General

Profile

Feature #13406

nexus drivers evade topo nic logic

Added by Robert Mustacchi 4 months ago. Updated 2 months ago.

Status:
Closed
Priority:
Normal
Category:
lib - userland libraries
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

NIC drivers vary in how they represent multiple-ports that are attached to the same physical PCI device. Many devices use a single PCI function per-port (regardless of whether they have shared resources or not) and therefore a single driver instance per function. Other drivers, like cxgbe, use a nexus driver to represent the device as a whole (t4nex in this case) and then enumerate a child device per-port that attaches to mac.

Right now the topo nic logic introduced in #8699 and #12206 assumes the per-function logic. We should add some additional logic in topo that can handle this and creates the same underlying abstraction in topo.

For example, this is what we now see for an instance of a Chelsio T6:

hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4
    label             string    MB
    FRU               fmri      hc://:server-id=beowulf/motherboard=0
    ASRU              fmri      dev:////pci@78,0/pci1022,1483@3,1/pci1425,0@0,4
  group: authority                      version: 1   stability: Private/Private
    server-id         string    beowulf
  group: io                             version: 1   stability: Private/Private
    dev               string    /pci@78,0/pci1022,1483@3,1/pci1425,0@0,4
    driver            string    t4nex
    instance          uint32    0x0
    module            fmri      mod:///mod-name=t4nex/mod-id=194
  group: pci                            version: 1   stability: Private/Private
    device-id         string    6408
    extended-capabilities string    pciexdev
    class-code        string    20000
    vendor-id         string    1425
    assigned-addresses uint32[]  [ 2206270480 0 4080009216 0 524288 2206270488 0 4060086272 0 16777216 2206270496 0 4086841344 0 8192 ]
    vendor-name       string    Chelsio Communications Inc
    device-name       string    T62100-SO-CR Unified Wire Ethernet Controller

hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=0
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=0
    FRU               fmri      hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4
  group: authority                      version: 1   stability: Private/Private
    server-id         string    beowulf
  group: port                           version: 1   stability: Private/Private
    type              string    sff
  group: datalink                       version: 1   stability: Private/Private
    link-speed        uint64    0x174876e800
    link-duplex       string    full
    link-status       string    up
    link-name         string    cxgbe0
    primary-mac-address string    00:07:43:5d:4d:60

hc://:server-id=beowulf:serial=F1940192980-2:part=Q28-PC01:revision=A/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=0/transceiver=0
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:server-id=beowulf:serial=F1940192980-2:part=Q28-PC01:revision=A/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=0/transceiver=0
    FRU               fmri      hc://:server-id=beowulf:serial=F1940192980-2:part=Q28-PC01:revision=A/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=0/transceiver=0
  group: authority                      version: 1   stability: Private/Private
    server-id         string    beowulf
  group: transceiver                    version: 1   stability: Private/Private
    type              string    sff
    usable            string    true
  group: sff-transceiver                version: 1   stability: Private/Private
    vendor            string    FS
    part-number       string    Q28-PC01
    revision          string    A
    serial-number     string    F1940192980-2

hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=1
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=1
    FRU               fmri      hc://:server-id=beowulf/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4
  group: authority                      version: 1   stability: Private/Private
    server-id         string    beowulf
  group: port                           version: 1   stability: Private/Private
    type              string    sff
  group: datalink                       version: 1   stability: Private/Private
    link-speed        uint64    0x174876e800
    link-duplex       string    full
    link-status       string    up
    link-name         string    cxgbe1
    primary-mac-address string    00:07:43:5d:4d:68
hc://:server-id=beowulf:serial=F1940192980-1:part=Q28-PC01:revision=A/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=1/transceiver=0
  group: protocol                       version: 1   stability: Private/Private
    resource          fmri      hc://:server-id=beowulf:serial=F1940192980-1:part=Q28-PC01:revision=A/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=1/transceiver=0
    FRU               fmri      hc://:server-id=beowulf:serial=F1940192980-1:part=Q28-PC01:revision=A/motherboard=0/hostbridge=8/pciexrc=8/pciexbus=129/pciexdev=0/pciexfn=4/port=1/transceiver=0
  group: authority                      version: 1   stability: Private/Private
    server-id         string    beowulf
  group: transceiver                    version: 1   stability: Private/Private
    type              string    sff
    usable            string    true
  group: sff-transceiver                version: 1   stability: Private/Private
    vendor            string    FS
    part-number       string    Q28-PC01
    revision          string    A
    serial-number     string    F1940192980-1

Note that the reason that the transceiver has the same information in both cases is because I had a cable looping back upon itself.

Also available in: Atom PDF