illumos gate

I've seen something similar to this on a bit of questionable hardware and have some ideas for the fix. Basically, the problem here is that while bringing up the boot CPU we assume that the bios always starts us in local apic mode. However, in this case the bios actually started the apic in x2apic mode. We don't detect that and appropriately swap over our function pointers to use the new programming model. This means that when we try to program the apic to do calibration, nothing happens. We do a good number of pit ticks, but no apic interrupts have come in and we blow up. There are two things that we should do here:

1) Make so if we actually get zero apic ticks that we panic right away because this means we have bad hardware (real or virtualized)
2) We need to properly detect what mode the bios has put the apic into on boot up, either local apic or x2apic, and set the mode appropriately.

I'm seeing this exact thing on my Lenovo RD450 machines also when x2APIC is enabled in the CMOS.

Note from Hans:

This can be fixed by switching the register access method in apix_probe() when we detect that the apic is already in x2apic mode. The pcplusmp module has the same problem, but here we should just fail the probe instead of using pcplusmp to drive the x2apic. That could only happen in case someone deliberately set apix_enable=0.

What this fix doesn't address is the apparent dependency of x2apic on VT-d for interrupt remapping. For that reason switching back to local apic mode would be preferable, but then startup of secondary CPUs fails. Switching back to local apic mode after waking up the other CPUs and then recreating the whole apic and interrupt setup doesn't seem viable.