bhyve vlapic should set TMR on intr accept
The way that bhyve has implemented the emulation for TMR state in the vlapic has been at odds with how it is architecturally defined. The Intel manual states:
The trigger mode register (TMR) indicates the trigger mode of the interrupt (see Figure 10-20). Upon acceptance of an interrupt into the IRR, the corresponding TMR bit is cleared for edge-triggered interrupts and set for level-triggered interrupts. If a TMR bit is set when an EOI cycle for its corresponding interrupt vector is generated, an EOI message is sent to all I/O APICs.
Rather than set/clear TMR bits when interrupts are accepted into IRR, bhyve sets them when a vioapic register is configured to deliver level-trigger interrupts to one or more vCPUs. The corresponding bits are permanently (for the life of that vioapic register state) asserted in the TMRs. While this does somewhat simplify interrupt delivery, particularly in the case of APICv, it does not properly represent how TMRs should be manipulated. It would be nice if TMR state was instead set during interrupt acceptance.
Updated by Patrick Mooney 12 months ago
Since this change overhauled event handling for both SVM and VMX, testing on both platforms is a must. I ran the standard set of OS smoke-tests on both platforms, booting up Linux, OmniOS, OpenBSD, and Windows. In the case of Intel hardware, I did so with APICv fully enabled, APICv disabled but TPR shadowing still on, and with all APIC-related acceleration disabled. To verify other event code paths, I confirmed that an injected NMI was handled by the guest. Level-triggered interrupts were exercised via the power-button (acpi) interrupt (sending SIGTERM to bhyve).
Updated by Patrick Mooney 11 months ago
I did also confirm that the exception injection paths were exercised. Guests such as Linux typically access some MSRs which are not explicitly wired in the hypervisor, so
#GP is injected for those accesses. I saw those fire on both AMD and Intel.
Updated by Electric Monk 11 months ago
- Status changed from New to Closed
- % Done changed from 0 to 100
commit c74a40a584c9d875009f725565896fd7e8ee38d6 Author: Patrick Mooney <email@example.com> Date: 2020-11-24T20:30:25.000Z 13007 bhyve vlapic should set TMR on intr accept 13106 clarify PPR transitions in bhyve vLAPIC 13132 VMX event injection can race in bhyve 13259 SVM event injection can race in bhyve Reviewed by: Robert Mustacchi <firstname.lastname@example.org> Reviewed by: Toomas Soome <email@example.com> Approved by: Dan McDonald <firstname.lastname@example.org>