Project

General

Profile

Bug #13165

FEC enabled port connected to i40e may not come online

Added by Paul Winder 8 days ago. Updated 7 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

Seen on a XXV710 25g card.
The i40e port is connected to a switch port which supports/advertises FEC. Before an attempt is made to configure the interface, the status appears good:

# dladm show-link
LINK        CLASS     MTU    STATE    BRIDGE     OVER
i40e4       phys      1500   up       --         --
i40e5       phys      1500   up       --         --

After creating an address:
# dladm show-link
LINK        CLASS     MTU    STATE    BRIDGE     OVER
i40e4       phys      1500   up       --         --
i40e5       phys      1500   down     --         --
# ipadm show-addr
ADDROBJ           TYPE     STATE        ADDR
lo0/v4            static   ok           127.0.0.1/8
i40e5/v4          static   inaccessible 192.168.40.2/24

I wrote dtrace script which dumped the struct returned from the i40e_aq_get_link_info call. Before attempting to configure the interface:
struct i40e_aqc_get_link_status {
    u16 command_flags = 0x1
    u8 phy_type = 0x21
    u8 link_speed = 0x40
    u8 link_info = 0xe1
    u8 an_info = 0xe0
    u8 ext_info = 0xe0
    u8 loopback = 0
    u16 max_frame_size = 0x2600
    u8 config = 0x5
    union  {
        struct  {
            u8 power_desc = 0
            u8 [4] reserved = [ 0, 0, 0, 0x4 ]
        }
        struct  {
            u8 [4] link_type = [ 0, 0, 0, 0 ]
            u8 link_type_ext = 0x4
        }
    }
}

and after
struct i40e_aqc_get_link_status {
    u16 command_flags = 0x1
    u8 phy_type = 0x21
    u8 link_speed = 0
    u8 link_info = 0xca
    u8 an_info = 0x80
    u8 ext_info = 0xe0
    u8 loopback = 0
    u16 max_frame_size = 0x5f2
    u8 config = 0x4
    union  {
        struct  {
            u8 power_desc = 0
            u8 [4] reserved = [ 0, 0, 0, 0x4 ]
        }
        struct  {
            u8 [4] link_type = [ 0, 0, 0, 0 ]
            u8 link_type_ext = 0x4
        }
    }
}

Apart from the values which indicate the link is no longer "up", config is of interest. Before change its bits indicated I40E_AQ_CONFIG_CRC_ENA and I40E_AQ_CONFIG_FEC_KR_ENA, after the interface was configured only I40E_AQ_CONFIG_CRC_ENA. The FEC related field is cleared.

It looks like it gets cleared when the flow control is set to none during i40e_chip_start

        /* For now, we always disable Ethernet Flow Control. */
        hw->fc.requested_mode = I40E_FC_NONE;
        rc = i40e_set_fc(hw, &err, B_TRUE);

i40e_set_fc calls i40e_aq_set_phy_config, but when it does so it clears out a "config" argument and then sets it with all the abilities it knows about. FEC is not one of those abilities it knows about and so it gets set to zero which disables FEC.


Related issues

Related to illumos gate - Feature #13166: Support en_fec_cap and adv_fec_cap link properties in i40eIn Progress

Actions

History

#1

Updated by Paul Winder 8 days ago

  • Related to Feature #13166: Support en_fec_cap and adv_fec_cap link properties in i40e added
#2

Updated by Paul Winder 8 days ago

  • Status changed from New to In Progress
#3

Updated by Paul Winder 7 days ago

The test scenario is fairly binary. As in the initial description, before the change creating an address on the link caused the link to go down.

After the change, the link stays up and is able to pass traffic. I would not expect this change to have an other impact, but I ran iperf across the 25Gb link and was able to get close to line rate in send and receive directions.

#4

Updated by Electric Monk 7 days ago

  • Gerrit CR set to 943

Also available in: Atom PDF