Project

General

Profile

Actions

Bug #13007

closed

bhyve vlapic should set TMR on intr accept

Added by Patrick Mooney about 1 year ago. Updated 11 months ago.

Status:
Closed
Priority:
Normal
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
bhyve
Gerrit CR:

Description

The way that bhyve has implemented the emulation for TMR state in the vlapic has been at odds with how it is architecturally defined. The Intel manual states:

The trigger mode register (TMR) indicates the trigger mode of the interrupt (see Figure 10-20). Upon acceptance of an interrupt into the IRR, the corresponding TMR bit is cleared for edge-triggered interrupts and set for level-triggered interrupts. If a TMR bit is set when an EOI cycle for its corresponding interrupt vector is generated, an EOI message is sent to all I/O APICs.

Rather than set/clear TMR bits when interrupts are accepted into IRR, bhyve sets them when a vioapic register is configured to deliver level-trigger interrupts to one or more vCPUs. The corresponding bits are permanently (for the life of that vioapic register state) asserted in the TMRs. While this does somewhat simplify interrupt delivery, particularly in the case of APICv, it does not properly represent how TMRs should be manipulated. It would be nice if TMR state was instead set during interrupt acceptance.


Related issues

Related to illumos gate - Bug #13259: SVM event injection can race in bhyveClosedPatrick Mooney

Actions
Related to illumos gate - Bug #13106: clarify PPR transitions in bhyve vLAPICClosedPatrick Mooney

Actions
Related to illumos gate - Bug #13132: VMX event injection can race in bhyveClosedPatrick Mooney

Actions
Has duplicate illumos gate - Bug #12956: bhyve should set TMR on intr acceptDuplicatePatrick Mooney

Actions
Actions #1

Updated by Electric Monk about 1 year ago

  • Gerrit CR set to 835
Actions #2

Updated by Patrick Mooney about 1 year ago

This will effectively supersede the change made in OS-7622

Actions #3

Updated by Patrick Mooney 12 months ago

  • Related to Bug #13259: SVM event injection can race in bhyve added
Actions #4

Updated by Patrick Mooney 12 months ago

Since this change overhauled event handling for both SVM and VMX, testing on both platforms is a must. I ran the standard set of OS smoke-tests on both platforms, booting up Linux, OmniOS, OpenBSD, and Windows. In the case of Intel hardware, I did so with APICv fully enabled, APICv disabled but TPR shadowing still on, and with all APIC-related acceleration disabled. To verify other event code paths, I confirmed that an injected NMI was handled by the guest. Level-triggered interrupts were exercised via the power-button (acpi) interrupt (sending SIGTERM to bhyve).

Actions #5

Updated by Patrick Mooney 11 months ago

I've repeated the OS smoke tests on Intel and AMD with a rebased copy of the change and it continues to behave as expected.

Actions #6

Updated by Patrick Mooney 11 months ago

Additionally, I've been using this wad (with several others) when developing an alternative bhyve userspace implementation. That testing has been done on an AMD Rome system using DEBUG bits.

Actions #7

Updated by Patrick Mooney 11 months ago

  • Related to Bug #13106: clarify PPR transitions in bhyve vLAPIC added
Actions #8

Updated by Patrick Mooney 11 months ago

  • Related to Bug #13132: VMX event injection can race in bhyve added
Actions #9

Updated by Patrick Mooney 11 months ago

I did also confirm that the exception injection paths were exercised. Guests such as Linux typically access some MSRs which are not explicitly wired in the hypervisor, so #GP is injected for those accesses. I saw those fire on both AMD and Intel.

Actions #10

Updated by Electric Monk 11 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit c74a40a584c9d875009f725565896fd7e8ee38d6

commit  c74a40a584c9d875009f725565896fd7e8ee38d6
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2020-11-24T20:30:25.000Z

    13007 bhyve vlapic should set TMR on intr accept
    13106 clarify PPR transitions in bhyve vLAPIC
    13132 VMX event injection can race in bhyve
    13259 SVM event injection can race in bhyve
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Actions #11

Updated by Patrick Mooney 10 months ago

  • Has duplicate Bug #12956: bhyve should set TMR on intr accept added
Actions

Also available in: Atom PDF