illumos gate

I did some testing on Robert's behalf using this patch combined with #13554. I used my lso tests to generate IPv6 traffic in various scenarios (primary interface, VLAN, VNIC, tagged VNIC) and then used a specific dtrace script to verify that IPv6 ULP checksum flag was being set.

Keep in mind that we don't expect to see LSO for IPv6 because it's gated behind missing IPv6 TSO emulation logic in mac-loopback (see #12731).

rpz@thunderhead:~/rpz-misc/illumos/net-tests$ pfexec ./lso-tests cxgbe0 cxgbe1
PASS [test_lso_ip4_primary<cxgbe0>]
PASS [test_lso_ip4_primary_vlan<cxgbe0>]
PASS [test_lso_ip4_vnic<cxgbe0>]
PASS [test_lso_ip4_vnic_vlan<cxgbe0>]
PASS [test_lso_ip4_mac_loopback<cxgbe0>]
PASS [test_lso_ip4_mac_loopback_vlan<cxgbe0>]

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% The following tests will elide the LSO check until the   %
% mac sw LSO is updated to handle IPv6 -- for now these    %
% tests only verify that traffic is sent, not LSO.         %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
PASS [test_lso_ip6_primary<cxgbe0>]
PASS [test_lso_ip6_primary_vlan<cxgbe0>]
PASS [test_lso_ip6_vnic<cxgbe0>]
PASS [test_lso_ip6_vnic_vlan<cxgbe0>]
PASS [test_lso_ip6_mac_loopback<cxgbe0>]
PASS [test_lso_ip6_mac_loopback_vlan<cxgbe0>]

rpz@thunderhead:~$ pfexec dtrace -Cqs /data/d/cxgbe-tx-cksum.d
^C
=== COUNTS ===
TYPE     VLAN     HW FLAGS                 PKT COUNT       
IPv4     --       --                       13
IPv4     9        --                       18
OTH      --       --                       38
OTH      9        --                       39
IPv6     9        --                       100
IPv6     --       --                       103
IPv4     --       HDR + FULL ULP           11389
IPv4     9        HDR + FULL ULP           11488
IPv4     --       LSO + HDR + FULL ULP     18502
IPv4     9        LSO + HDR + FULL ULP     19469
IPv6     --       FULL ULP                 87520
IPv6     9        FULL ULP                 87529

Finally, just to be sure we were exercising the IPv6 hardware emulation on mac-loopback I used the following DTrace one-liner.

rpz@thunderhead:~$ pfexec dtrace -qn 'mac_sw_cksum_ipv6:entry { this->ip6 = (ip6_t *)(args[0]->b_rptr + arg1); @[inet_ntoa6(&this->ip6->ip6_src), inet_ntoa6(&this->ip6->ip6_dst)] = count(); }'
^C

  fd00:0:1:4d::5                                      fd00:0:1:4d::4                                                19519
  fd00:0:1:4d::4                                      fd00:0:1:4d::5                                                75844

To test this, I manually had a workspace where I reverted the TSO disablement for IPv6 and pulled in my fix for #13554. With that in place I set up four main test scenarios with a pair of Chelsio T6 devices:

1. IPv4 on a primary interface
2. IPv6 on a primary interface
3. IPv4 on a VLAN tagged VNIC
4. IPv6 on a VLAN taged VNIC

With each of these I did basic network testing (e.g. iperf) to make sure that we were able to establish connections and correctly stream TCP data. I manually made sure that we were receiving large blocks with a combination of snoop and DTrace. I also verified that there was a perf improvement over not having TSO by using ndd to disable LSO at an IP level and comparing.