bhyve exits on APIC access
Testing a BE for #13795, booting into a BE caused bhyve to exit with the following:
[2021-06-02T14:20:15.602943000Z] INFO: zoneadmd/6789 on bob: (stream=stderr) vm exit reason VMX rip 0xfffffffffbaf03b6 inst_length 2 status 0 exit_reason 44 (APIC access) [2021-06-02T14:20:15.603164000Z] INFO: zoneadmd/6789 on bob: (stream=stderr) qualification 0x0000000000000204 inst_type 0 inst_error 0
This happened after the 'DEBUG enabled' message on the console (so after the kernel was loaded):
illumos Version zfs-crypto-dnode-0-g72d3a7deb1 64-bit DEBUG enabled [NOTICE: Zone halted]
Restarting the instance with the same BE, and it booted just fine after that, so I haven't been able to recreate it again, but wanted to capture it in case others hit it.
This was on joyent_20201118T153314Z, so it's not the absolutely most recent, but not terribly ancient either.
I can't be sure this was the same code as when it occurred, but after rebooting, the instruction in question does at least look plausible (specific instruction is noted with asterisks):
> 0xfffffffffbaf03b6::dis local_apic_read: pushq %rbp local_apic_read+1: movq %rsp,%rbp local_apic_read+4: pushq %rdi local_apic_read+5: movl %edi,%edi local_apic_read+7: subq $0x8,%rsp local_apic_read+0xb: movq -0x3bae5c52(%rip),%rax <apicadr> local_apic_read+0x12: leaq (%rax,%rdi,4),%rax ** local_apic_read+0x16: movl (%rax),%eax ** local_apic_read+0x18: leave local_apic_read+0x19: ret
Updated by Patrick Mooney 10 days ago
A qualification of
204 indicates a linear data read access at offset
204, which would not be a legal access:
Any FP/MMX/SSE access to an APIC register, or any access that touches bytes 4 through 15 of an APIC register may cause undefined behavior and must not be executed.
It's not clear why the guest would have attempted such an access. Without more detail about the system state under the exit, we cannot know for sure. We should probably make the APIC access handlers more robust to emit an expected error to the guest for such behavior, rather than taking an unhandleable exit.