Feature #12693
closed
Enable Forward Error Correction (FEC) configuration via dladm
100%
Description
We encountered an issue with a 100G Chelsio card in which the link would not come up, one of Cheslio's recommendation is to check FEC settings as there are some "buggy switches" which do not always play well. As a test I created a private property and was able to turn FEC off on the Chelsio card and the link came up and was fully functional.
Following on from that I decided to add a public property, "fec", to dladm linkprop set of commands. It can have one of 4 values:
none - FEC is not used
auto - Enable all FEC codes and let the device decide which to use
rs - Use Reed-Solomon coding
base-r - Use Base-R (aka FireCode) coding
An Example:
# dladm show-linkprop -p fec mlxcx0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE mlxcx0 fec rw rs auto none,auto,rs,base-r # dladm set-linkprop -p fec=none mlxcx0 # dladm show-linkprop -p fec mlxcx0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE mlxcx0 fec rw none auto none,auto,rs,base-r
An example of using "auto". Notice after setting auto, the show-linkprop displays the actual FEC coding in use.
# dladm show-linkprop -p fec mlxcx0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE mlxcx0 fec rw none auto none,auto,rs,base-r # dladm set-linkprop -p fec=auto mlxcx0 # dladm show-linkprop -p fec mlxcx0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE mlxcx0 fec rw rs auto none,auto,rs,base-r
Support for this has been added the mlxcx and cxgbe drivers. Tested on 25G and 100G Mellanox, 100G Chelsio
Updated by Paul Winder about 2 years ago
Feed back from the review has suggested there are two "fec" fields.
"en_fec_cap" - To set what is advertised, and displays what has been requested.
"adv_fec_cap" - Show what has been negotiated across the link.
Their use and semantics is based on the other "en_" and "adv_" properties.
Some examples:
# dladm show-linkprop mlxcx0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE mlxcx0 speed r- 10000 10000 -- mlxcx0 autopush rw -- -- -- mlxcx0 zone rw -- -- -- mlxcx0 duplex r- full full half,full mlxcx0 state r- up up up,down mlxcx0 adv_autoneg_cap r- 1 1 1,0 mlxcx0 mtu rw 1500 1500 22-10000 mlxcx0 flowctrl -- -- no no,tx,rx,bi mlxcx0 adv_fec_cap r- none auto none,auto,rs,base-r mlxcx0 en_fec_cap rw auto auto none,auto,rs,base-r mlxcx0 adv_100gfdx_cap r- 0 0 1,0 mlxcx0 en_100gfdx_cap r- 0 0 1,0 . .
and use cases:
# dladm set-linkprop -t -p en_fec_cap=auto cxgbe0 # dladm show-linkprop -p adv_fec_cap,en_fec_cap cxgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE cxgbe0 adv_fec_cap r- none auto none,auto,rs,base-r cxgbe0 en_fec_cap rw auto auto none,auto,rs,base-r # # dladm set-linkprop -t -p en_fec_cap=none,rs cxgbe0 # dladm show-linkprop -p adv_fec_cap,en_fec_cap cxgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE cxgbe0 adv_fec_cap r- rs auto none,auto,rs,base-r cxgbe0 en_fec_cap rw none,rs auto none,auto,rs,base-r # dladm set-linkprop -t -p en_fec_cap=rs cxgbe0 # dladm show-linkprop -p adv_fec_cap,en_fec_cap cxgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE cxgbe0 adv_fec_cap r- rs auto none,auto,rs,base-r cxgbe0 en_fec_cap rw rs auto none,auto,rs,base-r # dladm set-linkprop -t -p en_fec_cap=auto cxgbe0 # dladm show-linkprop -p adv_fec_cap,en_fec_cap cxgbe0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE cxgbe0 adv_fec_cap r- none auto none,auto,rs,base-r cxgbe0 en_fec_cap rw auto auto none,auto,rs,base-r
Updated by Electric Monk almost 2 years ago
- Status changed from In Progress to Closed
- % Done changed from 80 to 100
git commit d77e6e0f12d19668c0e9068c0fcd7a2123da5373
commit d77e6e0f12d19668c0e9068c0fcd7a2123da5373
Author: Paul Winder <paul@winder.uk.net>
Date: 2020-05-19T14:56:46.000Z
12693 Enable Forward Error Correction (FEC) configuration via dladm
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Robert Mustacchi <rm@fingolfin.org>
Approved by: Dan McDonald <danmcd@joyent.com>