Want Fortville TSO support
We should add TCP Segmentation Offloading (aka TSO, aka LSO) support to the i40e driver.
Test configuration 1¶
The changes were tested using a pair of J3301 nodes in SPC staging:
NIC: Intel XXV710 for 25GbE SFP28
running smartos PI with OS-5225 changes (v1.0.2 i40e driver)
NIC: Intel XXV710 for 25GbE SFP28
running stock smartos PI (v1.0.1 i40e driver)
The i40e interfaces on both systems are connected to a common network via 25gbs links.
iperf network testing scenarios
iperf testing will be performed using the following network parameters:
scenario 1: LSO disabled on system 1, 1500 MTU (regression)
scenario 2: LSO enabled on system 1, 1500 MTU
results: see attachments v101_1500mtu_25g_iperf.txt and v102_1500mtu_25g_iperf.txt for precise numbers - but in summary, the average throughput for the existing v1.0.1 driver was 7.93gps and the average throughput for the new v1.0.2 driver was 10.1gbps
scenario 3: LSO enabled on system 1, on both systems, two i40e interfaces combined via LACP, 1500 MTU
results: see attachments v101_1500mtu_25g_aggr_iperf.txt and v102_1500mtu_25g_aggr_iperf.txt for iperf output - but in summary, the average throughput for the existing v1.0.1 driver was 19.9gps and the average throughput for the new v1.0.2 driver was roughly 25.8gbps
For each of the above test scenarios, iperf testing was done first with system 2 as the sender (to establish a baseline using the stock i40e driver), and then with system 1 as the sender.
Memory Leak Testing¶
To test for leaks in the i40e driver, network load (driven by iperf) was run between system 1 (running with kmem_flags=0xf) to system 2 for several hours. Afterwards, I forced a crash dump on system 1 and ran "findleaks" on the dump. The only reported leaks were in the mpt_sas driver. I filed OS-6900 to track those leaks.
Test Configuration 2¶
In this test configuration, the i40e interface is connected back-to-back to a 10g interface on a separate system (no switch). This configuration was used to sanity test TX from i40e using jumbo frames.
onboard Intel X722 for 10GBASE-T
onboard Intel X540-AT2
scenario 1: LSO disabled on system 1, 9000 MTU
For test scenario 1 above, iperf was first on stock smartos bits (i.e. the existing v1.0.1 i40e driver), to establish a baseline. It was then run again with smartos bits that include the changes for LSO. (i.e. the v1.0.2 i40e driver).
Results: see v101_9000mtu.txt and v102_LSO_disabled_9000mtu.txt for captured iperf output, but the performance observed was almost identical between the two configurations, which was the expected result.
scenario 2: LSO enabled on system 1, 9000 MTU
Results: see v102_LSO_enabled_9000mtu.txt - results similar to above and system was stable.
Updated by Electric Monk over 3 years ago
- Status changed from New to Closed
commit 09aee6126f680324a9b019f9b4c77309dc611bf9 Author: Ryan Zezeski <firstname.lastname@example.org> Date: 2019-07-15T18:17:00.000Z 11356 Want Fortville TSO support 11357 want i40e multi-group support 11358 i40e_alloc_ring_mem() unwinds when it shouldn't 11359 Rework i40e transmit descriptor logic Portions contributed by: Rob Johnston <email@example.com> Reviewed by: Robert Mustacchi <firstname.lastname@example.org> Reviewed by: Patrick Mooney <email@example.com> Reviewed by: Randy Fishel <firstname.lastname@example.org> Approved by: Garrett D'Amore <email@example.com>