Project

General

Profile

Bug #12299

Chelsio 10G & 40G cards max out around 3Gbps

Added by Jason Matthews 8 days ago. Updated about 21 hours ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:

Description

Cards seem to have a buffer fill and start sending flow control pause frames. Working directly with Chelsio for resolution. Created ticket to track the issue.

At first they suggested adding the following to /etc/system, but it had no noticeable impact on performance

set ddi_msix_alloc_limit=32

History

#1

Updated by Jason Matthews 8 days ago

I am referring to the T580 and T520 cards

#2

Updated by Jason Matthews about 21 hours ago

Got this back from Chelsio...

Hi Jason,

Our Engineering is able to find below observations. Engineering is still investigating the issue. Will post progress tomorrow. JFI….we didn’t use your setup.


1. This could be an Rx issue seen only with AMD CPU.

2. With Linux peer, seeing line rate

when smartOS is the sender.

3. same low Rx performance is also seen on Intel, QLogic & Emulex 10G

adapters

4. Rx issue is not seen when OpenIndiana (with CPU affinity configured) is

installed on the same AMD setup.

5. Configuring CPU affinity using dladm on smartOS is not giving expected

results. Looking into this…

Command to configure CPU affinity.

#dladm set-linkprop -p cpus=0-11 cxgbe0

Also available in: Atom PDF