Feature #13449

cxgbe IPv6 checksum and LSO support

Added by Robert Mustacchi 3 months ago. Updated about 2 months ago.

driver - device drivers
Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


We should wire up IPv6 checksum offload and LSO support in cxgbe(7D). Thankfully all of the T4-T6 support this and in most cases it's just a few simple additional declarations.


Updated by Electric Monk 3 months ago

  • Gerrit CR set to 1165

Updated by Ryan Zezeski about 2 months ago

I did some testing on Robert's behalf using this patch combined with #13554. I used my lso tests to generate IPv6 traffic in various scenarios (primary interface, VLAN, VNIC, tagged VNIC) and then used a specific dtrace script to verify that IPv6 ULP checksum flag was being set.

Keep in mind that we don't expect to see LSO for IPv6 because it's gated behind missing IPv6 TSO emulation logic in mac-loopback (see #12731).

rpz@thunderhead:~/rpz-misc/illumos/net-tests$ pfexec ./lso-tests cxgbe0 cxgbe1
PASS [test_lso_ip4_primary<cxgbe0>]
PASS [test_lso_ip4_primary_vlan<cxgbe0>]
PASS [test_lso_ip4_vnic<cxgbe0>]
PASS [test_lso_ip4_vnic_vlan<cxgbe0>]
PASS [test_lso_ip4_mac_loopback<cxgbe0>]
PASS [test_lso_ip4_mac_loopback_vlan<cxgbe0>]

% The following tests will elide the LSO check until the   %
% mac sw LSO is updated to handle IPv6 -- for now these    %
% tests only verify that traffic is sent, not LSO.         %
PASS [test_lso_ip6_primary<cxgbe0>]
PASS [test_lso_ip6_primary_vlan<cxgbe0>]
PASS [test_lso_ip6_vnic<cxgbe0>]
PASS [test_lso_ip6_vnic_vlan<cxgbe0>]
PASS [test_lso_ip6_mac_loopback<cxgbe0>]
PASS [test_lso_ip6_mac_loopback_vlan<cxgbe0>]

rpz@thunderhead:~$ pfexec dtrace -Cqs /data/d/cxgbe-tx-cksum.d
=== COUNTS ===
TYPE     VLAN     HW FLAGS                 PKT COUNT       
IPv4     --       --                       13
IPv4     9        --                       18
OTH      --       --                       38
OTH      9        --                       39
IPv6     9        --                       100
IPv6     --       --                       103
IPv4     --       HDR + FULL ULP           11389
IPv4     9        HDR + FULL ULP           11488
IPv4     --       LSO + HDR + FULL ULP     18502
IPv4     9        LSO + HDR + FULL ULP     19469
IPv6     --       FULL ULP                 87520
IPv6     9        FULL ULP                 87529

Finally, just to be sure we were exercising the IPv6 hardware emulation on mac-loopback I used the following DTrace one-liner.

rpz@thunderhead:~$ pfexec dtrace -qn 'mac_sw_cksum_ipv6:entry { this->ip6 = (ip6_t *)(args[0]->b_rptr + arg1); @[inet_ntoa6(&this->ip6->ip6_src), inet_ntoa6(&this->ip6->ip6_dst)] = count(); }'

  fd00:0:1:4d::5                                      fd00:0:1:4d::4                                                19519
  fd00:0:1:4d::4                                      fd00:0:1:4d::5                                                75844

Updated by Robert Mustacchi about 2 months ago

To test this, I manually had a workspace where I reverted the TSO disablement for IPv6 and pulled in my fix for #13554. With that in place I set up four main test scenarios with a pair of Chelsio T6 devices:

1. IPv4 on a primary interface
2. IPv6 on a primary interface
3. IPv4 on a VLAN tagged VNIC
4. IPv6 on a VLAN taged VNIC

With each of these I did basic network testing (e.g. iperf) to make sure that we were able to establish connections and correctly stream TCP data. I manually made sure that we were receiving large blocks with a combination of snoop and DTrace. I also verified that there was a perf improvement over not having TSO by using ndd to disable LSO at an IP level and comparing.


Updated by Electric Monk about 2 months ago

  • Status changed from New to Closed
  • % Done changed from 90 to 100

git commit d75c60626217c1d02658d48c8cabf81bd44c6729

commit  d75c60626217c1d02658d48c8cabf81bd44c6729
Author: Robert Mustacchi <>
Date:   2021-02-24T20:14:51.000Z

    13449 cxgbe IPv6 checksum and LSO support
    Reviewed by: Ryan Zezeski <>
    Reviewed by: Paul Winder <>
    Approved by: Richard Lowe <>

Also available in: Atom PDF