illumos gate

ixgbe(7D)'s attach and initialization routine expects the PHY and SFP Tx optics (if present) to be powered on. I ran into an interesting case where, after rebooting from a Linux image into OmniOS with an in-development ixgbe for X550, my X550 failed to negotiate any link, and I could not force it to. Only after a system power cycle (power off & power on) would I resume getting a link.

It turns out that, as a part of Linux's ixgbe driver stop routine, it powers down the PHY and switches off any SFP Tx laser. The PHY will stay in this powered-off state until it is told to turn back on, which is not caused by a system reset or controller reset - one has to explicitly turn the hardware back on. If you do not do that, one will need to power-cycle the system to force the PHY back into its default-on state.

FreeBSD recently fixed this issue: https://github.com/freebsd/freebsd/commit/8887150af48686df7b5c2f7317a20a9d5320ab5f#diff-b0214db036b5b1c842cb85157352fdee