Bug #13007
bhyve vlapic should set TMR on intr accept
100%
Description
The way that bhyve has implemented the emulation for TMR state in the vlapic has been at odds with how it is architecturally defined. The Intel manual states:
The trigger mode register (TMR) indicates the trigger mode of the interrupt (see Figure 10-20). Upon acceptance of an interrupt into the IRR, the corresponding TMR bit is cleared for edge-triggered interrupts and set for level-triggered interrupts. If a TMR bit is set when an EOI cycle for its corresponding interrupt vector is generated, an EOI message is sent to all I/O APICs.
Rather than set/clear TMR bits when interrupts are accepted into IRR, bhyve sets them when a vioapic register is configured to deliver level-trigger interrupts to one or more vCPUs. The corresponding bits are permanently (for the life of that vioapic register state) asserted in the TMRs. While this does somewhat simplify interrupt delivery, particularly in the case of APICv, it does not properly represent how TMRs should be manipulated. It would be nice if TMR state was instead set during interrupt acceptance.
Related issues
Updated by Patrick Mooney 8 months ago
This will effectively supersede the change made in OS-7622
Updated by Patrick Mooney 6 months ago
- Related to Bug #13259: SVM event injection can race in bhyve added
Updated by Patrick Mooney 5 months ago
Since this change overhauled event handling for both SVM and VMX, testing on both platforms is a must. I ran the standard set of OS smoke-tests on both platforms, booting up Linux, OmniOS, OpenBSD, and Windows. In the case of Intel hardware, I did so with APICv fully enabled, APICv disabled but TPR shadowing still on, and with all APIC-related acceleration disabled. To verify other event code paths, I confirmed that an injected NMI was handled by the guest. Level-triggered interrupts were exercised via the power-button (acpi) interrupt (sending SIGTERM to bhyve).
Updated by Patrick Mooney 5 months ago
I've repeated the OS smoke tests on Intel and AMD with a rebased copy of the change and it continues to behave as expected.
Updated by Patrick Mooney 5 months ago
Additionally, I've been using this wad (with several others) when developing an alternative bhyve userspace implementation. That testing has been done on an AMD Rome system using DEBUG bits.
Updated by Patrick Mooney 5 months ago
- Related to Bug #13106: clarify PPR transitions in bhyve vLAPIC added
Updated by Patrick Mooney 5 months ago
- Related to Bug #13132: VMX event injection can race in bhyve added
Updated by Patrick Mooney 5 months ago
I did also confirm that the exception injection paths were exercised. Guests such as Linux typically access some MSRs which are not explicitly wired in the hypervisor, so #GP is injected for those accesses. I saw those fire on both AMD and Intel.
Updated by Electric Monk 5 months ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit c74a40a584c9d875009f725565896fd7e8ee38d6
commit c74a40a584c9d875009f725565896fd7e8ee38d6
Author: Patrick Mooney <pmooney@pfmooney.com>
Date: 2020-11-24T20:30:25.000Z
13007 bhyve vlapic should set TMR on intr accept
13106 clarify PPR transitions in bhyve vLAPIC
13132 VMX event injection can race in bhyve
13259 SVM event injection can race in bhyve
Reviewed by: Robert Mustacchi <rm@fingolfin.org>
Reviewed by: Toomas Soome <tsoome@me.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Updated by Patrick Mooney 4 months ago
- Has duplicate Bug #12956: bhyve should set TMR on intr accept added