Project

General

Profile

Actions

Bug #13847

open

bhyve should handle all APICv exits

Added by Jason King 12 months ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Category:
bhyve
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

Testing a BE for #13795, booting into a BE caused bhyve to exit with the following:

[2021-06-02T14:20:15.602943000Z]  INFO: zoneadmd/6789 on bob: (stream=stderr)
    vm exit[0]
        reason          VMX
        rip             0xfffffffffbaf03b6
        inst_length     2
        status          0
        exit_reason     44 (APIC access)

[2021-06-02T14:20:15.603164000Z]  INFO: zoneadmd/6789 on bob: (stream=stderr)
        qualification   0x0000000000000204
        inst_type               0
        inst_error              0

This happened after the 'DEBUG enabled' message on the console (so after the kernel was loaded):

illumos Version zfs-crypto-dnode-0-g72d3a7deb1 64-bit
DEBUG enabled

[NOTICE: Zone halted]

Restarting the instance with the same BE, and it booted just fine after that, so I haven't been able to recreate it again, but wanted to capture it in case others hit it.

This was on joyent_20201118T153314Z, so it's not the absolutely most recent, but not terribly ancient either.

I can't be sure this was the same code as when it occurred, but after rebooting, the instruction in question does at least look plausible (specific instruction is noted with asterisks):

> 0xfffffffffbaf03b6::dis
local_apic_read:                pushq  %rbp
local_apic_read+1:              movq   %rsp,%rbp
local_apic_read+4:              pushq  %rdi
local_apic_read+5:              movl   %edi,%edi
local_apic_read+7:              subq   $0x8,%rsp
local_apic_read+0xb:            movq   -0x3bae5c52(%rip),%rax   <apicadr>
local_apic_read+0x12:           leaq   (%rax,%rdi,4),%rax
** local_apic_read+0x16:           movl   (%rax),%eax **
local_apic_read+0x18:           leave
local_apic_read+0x19:           ret
Actions #1

Updated by Jason King 12 months ago

  • Description updated (diff)
Actions #2

Updated by Patrick Mooney 12 months ago

A qualification of 204 indicates a linear data read access at offset 204, which would not be a legal access:

Any FP/MMX/SSE access to an APIC register, or any access that touches bytes 4 through 15 of an APIC register may cause undefined behavior and must not be executed.

It's not clear why the guest would have attempted such an access. Without more detail about the system state under the exit, we cannot know for sure. We should probably make the APIC access handlers more robust to emit an expected error to the guest for such behavior, rather than taking an unhandleable exit.

Actions #3

Updated by Patrick Mooney about 1 month ago

  • Assignee set to Patrick Mooney
Actions #4

Updated by Patrick Mooney about 1 month ago

  • Subject changed from bhyve exits on APIC access to bhyve should handle all APICv exits
Actions #5

Updated by Patrick Mooney about 1 month ago

We should probably just fall back to the regular instruction emulation for cases like this.

Actions

Also available in: Atom PDF