Project

General

Profile

Feature #11359

Rework i40e transmit descriptor logic

Added by Robert Mustacchi 3 months ago. Updated 3 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
driver - device drivers
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:

Description

As part of the implementation of LSO, we discovered a number of issues with the original implementations and it took us a number of paths to get there. This change reworks the path to deal with a number of quirks of the i40e firmware and hardware which took a number of steps to get there. This includes a number of bugs that we've hit along the way, like:

Testing notes.

  • Compiled with gcc4 + gcc7 shadow.
  • Compiled DEBUG with gcc4.
  • Smatch clean.
  • Ran following tests on DEBUG kernel. All tests ran over VNICs on separate i40e ports. Verified traffic is hitting the wire via nicstat and also verified this modified LSO logic is being exercised via dtrace.
  • * GZ to bhyve
  • * * iperf3 default, TCP + UDP, VLAN + non-VLAN
  • * * iperf3 500b, TCP + UDP, VLAN + non-VLAN
  • * * iperf3 52b, TCP + UDP, VLAN + non-VLAN
  • * * mixed 52 + 301 byte packets, TCP, VLAN + non-VLAN
  • * bhyve to GZ
  • * * iperf3 default TCP + UDP, VLAN + non-VLAN
  • * * iperf3 500b TCP + UDP, VLAN + non-VLAN
  • * * iperf3 52b TCP + UDP, VLAN + non-VLAN
  • * * mixed 52 + 301 byte packets, TCP, VLAN + non-VLAN
  • * GZ to GZ
  • * * iperf3 default TCP + UDP, VLAN + non-VLAN
  • * * iperf3 500b TCP + UDP, VLAN + non-VLAN
  • * * iperf3 52b TCP + UDP, VLAN + non-VLAN
  • * * mixed 52 + 301 byte packets, TCP, VLAN + non-VLAN

This set of tests was run for each variation below. For each bullet point, if a parameter is not listed it means the default was used.

  • (default everything, non-DEBUG kernel)
  • over VLAN VNICs (there should be no difference from regular VNICs, but as a sanity check)
  • non-DEBUG + kmem flags (followed by NMI and ::findleaks)
  • _tx_dma_threshold=0
  • _tx_dma_threshold=10000
  • tx_lso_enable = 0
  • _tx_dma_threshold=0, tx_lso_enable = 0
  • _tx_dma_threshold=10000, tx_lso_enable = 0
  • 9000 MTU
  • _tx_dma_threshold=0, 9000 MTU
  • _tx_dma_threshold=10000, 9000 MTU
  • tx_lso_enable = 0, 9000 MTU
  • _tx_dma_threshold=0, tx_lso_enable = 0, 9000 MTU
  • _tx_dma_threshold=10000, tx_lso_enable = 0, 9000 MTU
  • tx_ring_size = 64
  • tx_ring_size = 64, _tx_dma_threshold=0
  • tx_ring_size = 64, _tx_dma_threshold=10000
  • DEBUG
  • DEBUG, _tx_dma_threshold=0
  • DEBUG, _tx_dma_threshold=10000

History

#1

Updated by Electric Monk 3 months ago

  • Status changed from New to Closed

git commit 09aee6126f680324a9b019f9b4c77309dc611bf9

commit  09aee6126f680324a9b019f9b4c77309dc611bf9
Author: Ryan Zezeski <rpz@joyent.com>
Date:   2019-07-15T18:17:00.000Z

    11356 Want Fortville TSO support
    11357 want i40e multi-group support
    11358 i40e_alloc_ring_mem() unwinds when it shouldn't
    11359 Rework i40e transmit descriptor logic
    Portions contributed by: Rob Johnston <rob.johnston@joyent.com>
    Reviewed by: Robert Mustacchi <rm@joyent.com>
    Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
    Reviewed by: Randy Fishel <randyf@sibernet.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

Also available in: Atom PDF