Project

General

Profile

Actions

Bug #16590

open

speed difference between IPv4 & IPv6 on 10Gbit connections

Added by Lee Damon 12 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
External Bug:

Description

This is basically a raw data collected by runnig iperf3 between two test hosts. They had no load on them at the time of the testing.

All of these tests were done between dedicated ixgbe cards. We see similar results with i40e interfaces but I can't test the same way as I don't have any spares. The i40e connections are all aggregated and go through a Juniper switch so I don't want to complicate things by adding them to this data dump, I just want to say I don't think this is limited to the ixgbe driver.

There was some suspicion the problem was caused by aggregation but based on these numbers I don't think that's the main problem.

These hosts are still configured for this test though hvfs2 was updated to latest release after these tests we run. I can run any additional tests needed/desired.

iperf3 was run as 'iperf3 -s' and 'iperf3 -t 60 -c ipaddress', no additional flaggs were used.

fs2:

SunOS fs2 5.11 omnios-r151050-6f87d0b5d63 i86pc i386 i86pc


hvfs2:
SunOS hvfs2 5.11 omnios-r151046-d2b54a0125 i86pc i386 i86pc


All tests done as direct connection between the two cards, no switch.

To create the non-aggregate interface (e.g. on hvfs2):

sudo ipadm create-addr -t -T static -a 10.64.18.123 ixgbe4/v4
sudo ipadm create-addr -t -T addrconf ixgbe4/v6
sudo ipadm create-addr -t -T static -a 2608::123/64 ixgbe4/v6a

same for fs2 but ixgbe0 and 121 instead of 123.

No aggregate
-------------

from fs2-test to hvfs2-test
iperf3 -t 60 -c 10.64.18.123

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  39.3 GBytes  5.63 Gbits/sec                  sender
[  4]   0.00-60.00  sec  39.3 GBytes  5.63 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  54.2 GBytes  7.76 Gbits/sec                  sender
[  4]   0.00-60.00  sec  54.2 GBytes  7.76 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  40.3 GBytes  5.78 Gbits/sec                  sender
[  4]   0.00-60.00  sec  40.3 GBytes  5.78 Gbits/sec                  receiver

iperf3 -t 60 -c 2608::123

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  29.9 GBytes  4.28 Gbits/sec                  sender
[  4]   0.00-60.00  sec  29.9 GBytes  4.28 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  26.6 GBytes  3.81 Gbits/sec                  sender
[  4]   0.00-60.00  sec  26.6 GBytes  3.81 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  27.4 GBytes  3.92 Gbits/sec                  sender
[  4]   0.00-60.00  sec  27.4 GBytes  3.92 Gbits/sec                  receiver

from hvfs2-test to fs2-test
iperf3 -t 60 -c 10.64.18.121

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  33.1 GBytes  4.74 Gbits/sec                  sender
[  4]   0.00-60.00  sec  33.1 GBytes  4.74 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  44.9 GBytes  6.43 Gbits/sec                  sender
[  4]   0.00-60.00  sec  44.9 GBytes  6.43 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  55.3 GBytes  7.92 Gbits/sec                  sender
[  4]   0.00-60.00  sec  55.3 GBytes  7.92 Gbits/sec                  receiver

iperf3 -t 60 -c 2608::121

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  30.7 GBytes  4.40 Gbits/sec                  sender
[  4]   0.00-60.00  sec  30.7 GBytes  4.40 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  30.5 GBytes  4.36 Gbits/sec                  sender
[  4]   0.00-60.00  sec  30.5 GBytes  4.36 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  33.8 GBytes  4.83 Gbits/sec                  sender
[  4]   0.00-60.00  sec  33.8 GBytes  4.83 Gbits/sec                  receiver

with aggregate:

To create the aggregate interface (e.g. on hvfs2):

sudo dladm create-aggr -L active -l ixgbe4 -l ixgbe5 aggr_test0
sudo ipadm create-addr -t -T static -a 10.64.18.123 aggr_test0/v4
sudo ipadm create-addr -t -T addrconf aggr_test0/v6
sudo ipadm create-addr -t -T static -a 2608::123/64 aggr_test0/v6a

same for fs2 but ixgbe0 & ixgbe1 and 121 instead of 123.

For the record:

: || lvd@fs2 ~ [615] ; dladm show-aggr -P | egrep "LINK|aggr_test0" 
LINK            POLICY   ADDRPOLICY           LACPACTIVITY  LACPTIMER   FLAGS
aggr_test0      L4       auto                 active        short       -----

: || lvd@hvfs2 ~ [584] ; dladm show-aggr -P | egrep "LINK|aggr_test0" 
LINK            POLICY   ADDRPOLICY           LACPACTIVITY  LACPTIMER   FLAGS
aggr_test0      L4       auto                 active        short       -----

from fs2-test to hvfs2-test
iperf3 -t 60 -c 10.64.18.123

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  45.8 GBytes  6.56 Gbits/sec                  sender
[  4]   0.00-60.00  sec  45.8 GBytes  6.56 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  49.5 GBytes  7.08 Gbits/sec                  sender
[  4]   0.00-60.00  sec  49.5 GBytes  7.08 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  34.2 GBytes  4.90 Gbits/sec                  sender
[  4]   0.00-60.00  sec  34.2 GBytes  4.90 Gbits/sec                  receiver

iperf3 -t 60 -c 2608::123

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  26.7 GBytes  3.82 Gbits/sec                  sender
[  4]   0.00-60.00  sec  26.7 GBytes  3.82 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  23.1 GBytes  3.31 Gbits/sec                  sender
[  4]   0.00-60.00  sec  23.1 GBytes  3.31 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  28.4 GBytes  4.06 Gbits/sec                  sender
[  4]   0.00-60.00  sec  28.4 GBytes  4.06 Gbits/sec                  receiver

from hvfs2-test to fs2-test
iperf3 -t 60 -c 10.64.18.121

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  50.4 GBytes  7.21 Gbits/sec                  sender
[  4]   0.00-60.00  sec  50.4 GBytes  7.21 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  41.1 GBytes  5.88 Gbits/sec                  sender
[  4]   0.00-60.00  sec  41.1 GBytes  5.88 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  30.8 GBytes  4.41 Gbits/sec                  sender
[  4]   0.00-60.00  sec  30.8 GBytes  4.41 Gbits/sec                  receiver

iperf3 -t 60 -c 2608::121

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  34.1 GBytes  4.88 Gbits/sec                  sender
[  4]   0.00-60.00  sec  34.1 GBytes  4.88 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  37.0 GBytes  5.29 Gbits/sec                  sender
[  4]   0.00-60.00  sec  37.0 GBytes  5.29 Gbits/sec                  receiver

[  4]   0.00-60.00  sec  32.8 GBytes  4.70 Gbits/sec                  sender
[  4]   0.00-60.00  sec  32.8 GBytes  4.70 Gbits/sec                  receiver

No data to display

Actions

Also available in: Atom PDF