Project

General

Profile

Actions

Bug #14235

closed

pcitool doesn't properly handle AMD Type 1 I/O space access

Added by Robert Mustacchi 6 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Category:
driver - device drivers
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

There is a PC engines APU that is based on AMD's Jaguar processor. The firmware on the PC engines board does not enable MMIO cfgspcae (though we could). While trying to use pcieadm on this, I found that it was always failing to read offsets at 0x100, suspiciously right where we transition to PCIe extended configuration space.

In pcitool_cfg_access we'll end up calling pci_cfgacc_acc which looks to see if we have MMIO configuration space and if not fall back to an I/O port access. This is all well and good; however, after doing this, there is a check that sees if we would have read beyond the maximum size of I/O space. There is also one earlier in the function, but because we enter with a MMIO access read it doesn't think about it. The fundamental problem here is that this is not using the actual kernel's determination of what the maximum I/O space mechanism is, it just assumes it's 0xff. The solution here is to use the pci_iocfg_max_offset value.

Actions #1

Updated by Electric Monk 6 months ago

  • Gerrit CR set to 1799
Actions #2

Updated by Robert Mustacchi 6 months ago

I tested this by using pcieadm. Previously it would fail on the rge0 devices in this PC Engines APU; however, now it succeeds and displays all of configuration space for the device. For example:

rm@lobster ~ $ pfexec /usr/lib/pci/pcieadm show-cfgspace -d 1/0/0
Device 1/0/0 -- Type 0 Header
  Vendor ID: 0x10ec -- Realtek Semiconductor Co., Ltd.
  Device ID: 0x8168 -- RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
  Command: 0x47
    |--> I/O Space: enabled (0x1)
    |--> Memory Space: enabled (0x2)
    |--> Bus Master: enabled (0x4)
    |--> Special Cycle: disabled (0x0)
    |--> Memory Write and Invalidate: disabled (0x0)
    |--> VGA Palette Snoop: disabled (0x0)
    |--> Parity Error Response: enabled (0x40)
    |--> IDSEL Stepping/Wait Cycle Control: unsupported (0x0)
    |--> SERR# Enable: disabled (0x0)
    |--> Fast Back-to-Back Transactions: disabled (0x0)
    |--> Interrupt X: enabled (0x0)
  Status: 0x10
    |--> Immediate Readiness: unsupported (0x0)
    |--> Interrupt Status: not pending (0x0)
    |--> Capabilities List: supported (0x10)
    |--> 66 MHz Capable: unsupported (0x0)
    |--> Fast Back-to-Back Capable: unsupported (0x0)
    |--> Master Data Parity Error: no error (0x0)
    |--> DEVSEL# Timing: fast (0x0)
    |--> Signaled Target Abort: no (0x0)
    |--> Received Target Abort: no (0x0)
    |--> Received Master Abort: no (0x0)
    |--> Signaled System Error: no (0x0)
    |--> Detected Parity Error: no (0x0)
  Revision ID: 0x6
  Class Code: 0x20000
    |--> Class Code: 0x2
    |--> Sub-Class Code: 0x0
    |--> Programming Interface: 0x0
  Cache Line Size: 0x40 bytes
  Latency Timer: 0x0 cycles
  Header Type: 0x0
    |--> Header Layout: Device (0x0)
    |--> Multi-Function Device: no (0x0)
  BIST: 0x0
    |--> Completion Code: 0x0
    |--> Start BIST: 0x0
    |--> BIST Capable: unsupported (0x0)
  Base Address Register 0
    |--> Space: I/O Space (0x1)
    |--> Address: 0x1000
  Base Address Register 1
    |--> Space: Memory Space (0x0)
    |--> Address: 0x0
    |--> Address: 32-bit (0x0)
    |--> Prefetchable: no (0x0)
  Base Address Register 2
    |--> Space: Memory Space (0x0)
    |--> Address: 0xf7a00000
    |--> Address: 64-bit (0x2)
    |--> Prefetchable: no (0x0)
  Base Address Register 4
    |--> Space: Memory Space (0x0)
    |--> Address: 0xf7900000
    |--> Address: 64-bit (0x2)
    |--> Prefetchable: yes (0x1)
  Cardbus CIS Pointer: 0x0
  Subsystem Vendor ID: 0x10ec -- Realtek Semiconductor Co., Ltd.
  Subsystem Device ID: 0x123
  Expansion ROM: 0x0
    |--> Enable: disabled (0x0)
    |--> Base Address: 0x0
  Capabilities Pointer: 0x40
  Interrupt Line: 0xa
  Interrupt Pin: 0x1 -- INTA
  Min_Gnt: 0x0
  Min_Lat: 0x0
PCI Power Management Capability (0x1)
  Power Management Capabilities: 0xffc3
    |--> Version: 0x3
    |--> PME Clock: not required (0x0)
    |--> Immediate Readiness on Return to D0: no (0x0)
    |--> Device Specific Initialization: no (0x0)
    |--> Auxiliary Current: 375 mA (0x1c0)
    |--> D1: supported (0x200)
    |--> D2: supported (0x400)
    |--> PME Support: 0xf800
      |--> D0 (0x800)
      |--> D1 (0x1000)
      |--> D2 (0x2000)
      |--> D3hot (0x4000)
      |--> D3cold (0x8000)
Message Signaled Interrupts Capability (0x5)
  Message Control: 0x81
    |--> MSI Enable: enabled (0x1)
    |--> Multiple Message Capable: 1 vector (0x0)
    |--> Multiple Message Enabled: 1 vector (0x0)
    |--> 64-bit Address Capable: supported (0x80)
    |--> Per-Vector Masking Capable: unsupported (0x0)
    |--> Extended Message Data Capable: unsupported (0x0)
    |--> extended Message Data Enable: unsupported (0x0)
  Message Address: 0xfee01000
  Upper Message Address: 0x0
  Message Data: 0x24
PCI Express Capability (0x10)
  Capability Register: 0x202
    |--> Version: 0x2
    |--> Device/Port Type: PCIe Endpoint (0x0)
    |--> Slot Implemented: No (0x0)
    |--> Interrupt Message Number: 0x1
  Device Capabilities: 0x8cc0
    |--> Max Payload Size Supported: 128 bytes (0x0)
    |--> Phantom Functions Supported: No (0x0)
    |--> Extended Tag Field: 5-bit (0x0)
    |--> L0s Acceptable Latency: 512 ns (0xc0)
    |--> L1 Acceptable Latency: 64 us (0xc00)
    |--> Role Based Error Reporting: supported (0x8000)
    |--> ERR_COR Subclass: unsupported (0x0)
    |--> Captured Slot Power Limit: 0x0
    |--> Captured Slot Power Limit Scale: 1.0x (0x0)
    |--> Function Level Reset: unsupported (0x0)
  Device Control: 0x2017
    |--> Correctable Error Reporting: enabled (0x1)
    |--> Non-Fatal Error Reporting: enabled (0x2)
    |--> Fatal Error Reporting: enabled (0x4)
    |--> Unsupported Request Reporting: disabled (0x0)
    |--> Relaxed Ordering: enabled (0x10)
    |--> Max Payload Size: 128 bytes (0x0)
    |--> Extended Tag Field: disabled (0x0)
    |--> Phantom Functions: disabled (0x0)
    |--> Aux Power PM: disabled (0x0)
    |--> No Snoop: disabled (0x0)
    |--> Max Read Request Size: 512 bytes (0x2000)
    |--> Bridge Configuration Retry / Function Level Reset: 0x0
  Device Status: 0x10
    |--> Correctable Error Detected: no (0x0)
    |--> Non-Fatal Error Detected: no (0x0)
    |--> Fatal Error Detected: no (0x0)
    |--> Unsupported Request Detected: no (0x0)
    |--> AUX Power Detected: yes (0x10)
    |--> Transactions Pending: no (0x0)
    |--> Emergency Power Reduction Detected: no (0x0)
  Link Capabilities: 0x73c11
    |--> Maximum Link Speed: 2.5 GT/s (0x1)
    |--> Maximum Link Width: 0x1
    |--> ASPM Support: L0s/L1 (0xc00)
    |--> L0s Exit Latency: 256-512ns (0x3000)
    |--> L1 Exit Latency: >64us (0x30000)
    |--> Clock Power Management: supported (0x40000)
    |--> Surprise Down Error Reporting: unsupported (0x0)
    |--> Data Link Layer Active Reporting: unsupported (0x0)
    |--> Link Bandwidth Notification Capability: unsupported (0x0)
    |--> ASPM Optionality Compliance: not compliant (0x0)
    |--> Port Number: 0x0
  Link Control: 0x3
    |--> ASPM Control: L0s/L1 (0x3)
    |--> Read Completion Boundary: 64 byte (0x0)
    |--> Link Disable: not force disabled (0x0)
    |--> Retrain Link: 0x0
    |--> Common Clock Configuration: asynchronous (0x0)
    |--> Extended Sync: 0x411f16
    |--> Clock Power Management: 0x411f16
    |--> Hardware Autonomous Width: 0x411ad8
    |--> Link Bandwidth Management Interrupt: 0x411f16
    |--> Link Autonomous Bandwidth Interrupt: 0x411f16
    |--> DRS Signaling Control: 0x4101f0
  Link Status: 0x1011
    |--> Link Speed: 2.5 GT/s (0x1)
    |--> Link Width: 0x1
    |--> Link Training: no (0x0)
    |--> Slot Clock Configuration: common (0x1000)
    |--> Data Link Layer Link Active: no (0x0)
    |--> Link Bandwidth Management Status: no change (0x0)
    |--> Link Autonomous Bandwidth Status: no change (0x0)
  Slot Capabilities: 0x0
    |--> Attention Button Present: no (0x0)
    |--> Power Controller Present: no (0x0)
    |--> MRL Sensor Present: no (0x0)
    |--> Attention Indicator Present: no (0x0)
    |--> Power Indicator Present: no (0x0)
    |--> Hot-Plug Surprise: unsupported (0x0)
    |--> Hot-Plug Capable : unsupported (0x0)
    |--> Slot Power Limit Value: 0x0
    |--> Slot Power Limit Scale: 0x0
    |--> Electromechanical Interlock Present: no (0x0)
    |--> No Command Completed: 0x41159e
    |--> Physical Slot Number: 0x0
  Slot Control: 0x0
    |--> Attention Button Pressed: disabled (0x0)
    |--> Power Fault Detected: disabled (0x0)
    |--> MRL Sensor Changed: disabled (0x0)
    |--> Presence Detect Changed: disabled (0x0)
    |--> Command Complete Interrupt: disabled (0x0)
    |--> Hot Plug Interrupt Enable: disabled (0x0)
    |--> Attention Indicator Control: reserved (0x0)
    |--> Power Indicator Control: reserved (0x0)
    |--> Power Controller Control: power on (0x0)
    |--> Electromechanical Interlock Control: 0x0
    |--> Data Link Layer State Changed: disabled (0x0)
    |--> Auto Slot Power Limit: enabled (0x0)
    |--> In-Band PD: enabled (0x0)
  Slot Status: 0x0
    |--> Attention Button Pressed: no (0x0)
    |--> Power Fault Detected: no (0x0)
    |--> MRL Sensor Changed: no (0x0)
    |--> Presence Detect Changed: no (0x0)
    |--> Command Complete: no (0x0)
    |--> MRL Sensor State: closed (0x0)
    |--> Presence Detect State: not present (0x0)
    |--> Electromechanical Interlock: disengaged (0x0)
    |--> Data Link Layer State Changed: no (0x0)
  Root Control: 0x0
    |--> CRS Software Visibility: disabled (0x0)
  Root Capabilities: 0x0
    |--> System Error on Correctable Error: disabled (0x0)
    |--> System Error on Non-Fatal Error: disabled (0x0)
    |--> System Error on Fatal Error: disabled (0x0)
    |--> PME Interrupt: disabled (0x0)
    |--> CRS Software Visibility: disabled (0x0)
  Root Status: 0x0
    |--> PME Requester ID: 0x0
    |--> PME Status: deasserted (0x0)
    |--> PME Pending: no (0x0)
  Device Capabilities 2: 0x10
    |--> Completion Timeout Ranges Supported: 0x0
    |--> Completion Timeout Disable: supported (0x10)
    |--> ARI Forwarding: unsupported (0x0)
    |--> AtomicOp Routing: unsupported (0x0)
    |--> 32-bit AtomicOp Completer: unsupported (0x0)
    |--> 64-bit AtomicOp Completer: unsupported (0x0)
    |--> 128-bit CAS Completer: unsupported (0x0)
    |--> No Ro-enabld PR-PR Passing: unsupported (0x0)
    |--> LTR Mechanism: unsupported (0x0)
    |--> TPH Completer: unsupported (0x0)
    |--> LN System CLS: unsupported (0x0)
    |--> 10-bit Tag Completer: unsupported (0x0)
    |--> 10-bit Tag Requester: unsupported (0x0)
    |--> OBFF: unsupported (0x0)
    |--> Extended Fmt Field Supported: unsupported (0x0)
    |--> End-End TLP Prefix Supported: unsupported (0x0)
    |--> Max End-End TLP Prefixes: 4 (0x0)
    |--> Emergency Power Reduction: unsupported (0x0)
    |--> Emergency Power Reduction Initialization Required: no (0x0)
    |--> Function Readiness Status: unsupported (0x0)
  Device Control 2: 0x0
    |--> Completion Timeout: 50us-50ms (0x0)
    |--> Completion Timeout Disabled: not disabled (0x0)
    |--> ARI Forwarding: disabled (0x0)
    |--> AtomicOp Requester: disabled (0x0)
    |--> AtomicOp Egress Blocking: unblocked (0x0)
    |--> ID-Based Ordering Request: disabled (0x0)
    |--> ID-Based Ordering Completion: disabled (0x0)
    |--> LTR Mechanism: disabled (0x0)
    |--> Emergency Power Reduction: not requested (0x0)
    |--> 10-bit Tag Requester: disabled (0x0)
    |--> OBFF: disabled (0x0)
    |--> End-End TLP Prefix Blocking: unblocked (0x0)
  Device Status 2: 0x0
  Link Capabilities 2: 0x0
    |--> Supported Link Speeds: 0x0
    |--> Crosslink: unsupported (0x0)
    |--> Lower SKP OS Generation Supported Speeds Vector: 0x0
    |--> Lower SKP OS Reception Supported Speeds Vector: 0x0
    |--> Retimer Presence Detect Supported: unsupported (0x0)
    |--> Two Retimers Presence Detect Supported: unsupported (0x0)
    |--> Device Readiness Status: unsupported (0x0)
  Link Control 2: 0x0
    |--> Target Link Speed: reserved (0x0)
    |--> Enter Compliance: no (0x0)
    |--> Hardware Autonomous Speed Disable: not disabled (0x0)
    |--> Selectable De-emphasis: -6 dB (0x0)
    |--> TX Margin: 0x0
    |--> Enter Modified Compliance: no (0x0)
    |--> Compliance SOS: disabled (0x0)
    |--> Compliance Preset/De-emphasis: 0x0
  Link Status 2: 0x0
    |--> Current De-emphasis Level: -6 dB (0x0)
    |--> Equalization 8.0 GT/s Complete: no (0x0)
    |--> Equalization 8.0 GT/s Phase 1: unsuccessful (0x0)
    |--> Equalization 8.0 GT/s Phase 2: unsuccessful (0x0)
    |--> Equalization 8.0 GT/s Phase 3: unsuccessful (0x0)
    |--> Link Equalization Request 8.0 GT/s: not requested (0x0)
    |--> Retimer Presence Detected: no (0x0)
    |--> Two Retimers Presence Detected: no (0x0)
    |--> Crosslink Resolution: unsupported (0x0)
    |--> Downstream Component Presence: link down - undetermined (0x0)
    |--> DRS Message Received: no (0x0)
  Slot Capabilities 2: 0x0
    |--> In-Band PD Disable: unsupported (0x0)
  Slot Control 2: 0x0
  Slot Status 2: 0x0
MSI-X Capability (0x11)
  Control Register: 0x3
    |--> Table Size: 0x4
    |--> Function Mask: unmasked (0x0)
    |--> MSI-X Enable: disabled (0x0)
  Table Offset: 0x4
    |--> Table BIR: BAR 4 (0x4)
    |--> Table Offset: 0x0
  PBA Offset: 0x804
    |--> PBA BIR: BAR 4 (0x4)
    |--> PBA Offset: 0x800
Vital Product Data Capability (0x3)
  VPD Address Register: 0x0
    |--> VPD Address: 0x0
    |--> Flag: 0x0
  VPD Data: 0x0
Advanced Error Reporting Capability (0x1)
  Capability Header: 0x14010001
    |--> Capability ID: 0x1
    |--> Capability Version: 0x1
    |--> Next Capability Offset: 0x140
  Uncorrectable Error Status: 0x0
    |--> Data Link Protocol Error: 0x0
    |--> Surprise Down Error: 0x0
    |--> Poisoned TLP Received: 0x0
    |--> Flow Control Protocol Error: 0x0
    |--> Completion Timeout: 0x0
    |--> Completion Abort: 0x0
    |--> Unexpected Completion: 0x0
    |--> Receiver Overflow: 0x0
    |--> Malformed TLP: 0x0
    |--> ECRC Error: 0x0
    |--> Unsupported Request Error: 0x0
    |--> ACS Violation: 0x0
    |--> Uncorrectable Internal Error: 0x0
    |--> MC Blocked TLP: 0x0
    |--> AtomicOp Egress Blocked: 0x0
    |--> TLP Prefix Blocked Error: 0x0
    |--> Poisoned TLP Egress Blocked: 0x0
  Uncorrectable Error Mask: 0x100000
    |--> Data Link Protocol Error: 0x0
    |--> Surprise Down Error: 0x0
    |--> Poisoned TLP Received: 0x0
    |--> Flow Control Protocol Error: 0x0
    |--> Completion Timeout: 0x0
    |--> Completion Abort: 0x0
    |--> Unexpected Completion: 0x0
    |--> Receiver Overflow: 0x0
    |--> Malformed TLP: 0x0
    |--> ECRC Error: 0x0
    |--> Unsupported Request Error: 0x1
    |--> ACS Violation: 0x0
    |--> Uncorrectable Internal Error: 0x0
    |--> MC Blocked TLP: 0x0
    |--> AtomicOp Egress Blocked: 0x0
    |--> TLP Prefix Blocked Error: 0x0
    |--> Poisoned TLP Egress Blocked: 0x0
  Uncorrectable Error Severity: 0x62030
    |--> Data Link Protocol Error: 0x1
    |--> Surprise Down Error: 0x1
    |--> Poisoned TLP Received: 0x0
    |--> Flow Control Protocol Error: 0x1
    |--> Completion Timeout: 0x0
    |--> Completion Abort: 0x0
    |--> Unexpected Completion: 0x0
    |--> Receiver Overflow: 0x1
    |--> Malformed TLP: 0x1
    |--> ECRC Error: 0x0
    |--> Unsupported Request Error: 0x0
    |--> ACS Violation: 0x0
    |--> Uncorrectable Internal Error: 0x0
    |--> MC Blocked TLP: 0x0
    |--> AtomicOp Egress Blocked: 0x0
    |--> TLP Prefix Blocked Error: 0x0
    |--> Poisoned TLP Egress Blocked: 0x0
  Correctable Error Status: 0x0
    |--> Receiver Error: 0x0
    |--> Bad TLP: 0x0
    |--> Bad DLLP: 0x0
    |--> REPLAY_NUM Rollover: 0x0
    |--> Replay timer Timeout: 0x0
    |--> Advisory Non-Fatal Error: 0x0
    |--> Correctable Internal Error: 0x0
    |--> Header Log Overflow: 0x0
  Correctable Error Mask: 0x0
    |--> Receiver Error: 0x0
    |--> Bad TLP: 0x0
    |--> Bad DLLP: 0x0
    |--> REPLAY_NUM Rollover: 0x0
    |--> Replay timer Timeout: 0x0
    |--> Advisory Non-Fatal Error: 0x0
    |--> Correctable Internal Error: 0x0
    |--> Header Log Overflow: 0x0
  Advanced Error Capabilities and Control: 0x1e0
    |--> First Error Pointer: 0x0
    |--> ECRC Generation Capable: supported (0x20)
    |--> ECRC Generation Enable: enabled (0x40)
    |--> ECRC Check Capable: supported (0x80)
    |--> ECRC Check Enable: enabled (0x100)
  Header Log 0: 0x0
  Header Log 1: 0x0
  Header Log 2: 0x0
  Header Log 3: 0x0
  Root Error Command: 0x1e0
    |--> Correctable Error Reporting: disabled (0x0)
    |--> Non-Fatal Error Reporting: disabled (0x0)
    |--> Fatal Error Reporting: disabled (0x0)
  Root Error Status: 0x0
    |--> ERR_COR Received: 0x0
    |--> Multiple ERR_COR Received: 0x0
    |--> ERR_FATAL/NONFATAL Received: 0x0
    |--> Multiple ERR_FATAL/NONFATAL Received: 0x0
    |--> First Uncorrectable Fatal: 0x0
    |--> Non-Fatal Error Messages Received: 0x0
    |--> Fatal Error Messages Received: 0x0
    |--> ERR_COR Subclass: ECS Legacy (0x0)
    |--> Advanced Error Interrupt Message: 0x0
  Error Source Identification: 0x0
    |--> ERR_COR Source: 0x0
    |--> ERR_FATAL/NONFATAL Source: 0x0
Virtual Channel Capability (0x2)
  Capability Header: 0x16010002
    |--> Capability ID: 0x2
    |--> Capability Version: 0x1
    |--> Next Capability Offset: 0x160
  Port VC Capability 1: 0x0
    |--> Extended VC Count: 0x0
    |--> Low Priority Extended VC Count: 0x0
    |--> Reference Clock: 100ns (0x0)
    |--> Port Arbitration Table Size: 1 bit (0x0)
  Port VC Capability 2: 0x0
    |--> VC Arbitration Capability: 0x0
    |--> VC Arbitration Table Offset: 0x0
  Port VC Control: 0x0
    |--> Load VC Arbitration Table: 0x0
    |--> VC Arbitration Select: hardware fixed (0x0)
  Port VC Status: 0x0
    |--> VC Arbitration Table Status: 0x0
  VC Resource 0 Capability: 0x0
    |--> Port Arbitration Capability: 0x0
    |--> Advanced Packet Switching: unsupported (0x0)
    |--> Reject Snoop Transactions: unsupported (0x0)
    |--> Maximum Time Slots: 0x1
    |--> VC Arbitration Table Offset: 0x0
  VC Resource 0 Control: 0x800000ff
    |--> TC/VC Map: 0xff
    |--> Load VC Arbitration Table: 0x0
    |--> Port Arbitration Select: hardware fixed (0x0)
    |--> VC ID: 0x0
    |--> VC Enable: enabled (0x80000000)
  VC Resource 0 Status: 0x0
    |--> Port Arbitration Table Status: 0x0
Serial Number Capability (0x3)
  Capability Header: 0x10003
    |--> Capability ID: 0x3
    |--> Capability Version: 0x1
    |--> Next Capability Offset: 0x0
  Serial Number: 01-00-00-00-68-4c-e0-00
Actions #3

Updated by Robert Mustacchi 6 months ago

I also did a few spot checks of other systems with pcieadm and made sure that we were still dumping configuration space correctly and that devices had not changed in their enumeration.

Actions #4

Updated by Electric Monk 6 months ago

  • Status changed from New to Closed
  • % Done changed from 90 to 100

git commit 671e12f2cf68dbd241dfc08296dc05fecc27127c

commit  671e12f2cf68dbd241dfc08296dc05fecc27127c
Author: Robert Mustacchi <rm@fingolfin.org>
Date:   2021-11-20T17:51:34.000Z

    14235 pcitool doesn't properly handle AMD Type 1 I/O space access
    Reviewed by: Andy Fiddaman <andy@omnios.org>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Rich Lowe <richlowe@richlowe.net>

Actions

Also available in: Atom PDF