i40e does not support Intel X722 + X557 PHY
I have a superMicro X11SPH-nCTF(1) and the onboard nics don't work.
They are in an unknown state when coming from a cold boot:
[root@boron ~]# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE i40e0 Ethernet unknown 0 unknown i40e0 i40e1 Ethernet unknown 0 unknown i40e1 ixgbe0 Ethernet up 10000 full ixgbe0 igb0 Ethernet up 1000 full igb0
The switch seems the as negotiated to 1Gbit FDX (which is correct), when resetting the box once it is been up. Both come up as 1000 full as expected.
Looking at the source(3) and X557 PHYs spec sheet (2) this makes sense, the device from cold boot will start in extended SGMII auto-negotiation, when looking a the code this ends up in the unknown state. After a reset while the links are up they are in the 1000BASE-X interface on (SGMII without auto-negotiation) state, so we do end up in the correct driver state!
From the X557 PHY linked below:
1.3 SerDes The X557 is designed to be used in conjunction with a triple-speed MAC chip: • KR/XFI for 10 GbE • 1000BASE-X for 1 GbE (SerDes logical lane 0) • SGMII for 100 Mb/s (SerDes logical lane 0) Note: The X557-AT2 10 GbE PHY internally has four SerDes per PHY, but only physical lane 2 is connected. Consequently, the register map contains information for lanes 0, 1 and 3, but these cannot be used in the X557-AT2. The X557 is capable of being configured so that the MAC interface can start in one of the following modes: 1. Start with the 10 GbE interface on KR/XFI. 2. Start with the 1000BASE-X interface on (SGMII without auto-negotiation). 3. Start with extended SGMII auto-negotiation enabled. 4. Start with all interfaces off. Once a connection to a link partner has been established, the interface switches to the correct MAC interface speed.
Although a X722 with SPF+ PHY doesn't seem to support 100 FDX/SGMII... with a X557 PHY attached it does seem to end up in this mode.
Forcing the switch to 100M FDX makes the link also come up, but after a reset the box remains in an unknown state... I'm not sure what I need to change to make it work from a cold boot.