Project

General

Profile

Actions

Bug #12991

closed

bhyve vlapic should SIPI more carefully

Added by Patrick Mooney 11 months ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
bhyve
Gerrit CR:

Description

Certain writes to the ICR LAPIC register can initiate transmission of a startup IPI (SIPI) to one of the other application processors (CPUs) on the system. Under bhyve, this requires an exit out to userspace in order to spin up a handler thread for the newly-started vCPU. The handler function which dispatches that information, vlapic_icrlo_write_handler(), does so by directly writing to the vCPU exitinfo field, requesting that the vCPU exit out to userspace:

        if (mode == APIC_DELMODE_STARTUP) {
                if (vlapic->vcpuid == 0 && dest != 0 && dest < maxcpus) {
                        vlapic2 = vm_lapic(vlapic->vm, dest);

                        /*
                         * Ignore SIPIs in any state other than wait-for-SIPI
                         */
                        if (vlapic2->boot_state != BS_SIPI)
                                return (0);

                        vlapic2->boot_state = BS_RUNNING;

                        *retu = true;
                        vmexit = vm_exitinfo(vlapic->vm, vlapic->vcpuid);
                        vmexit->exitcode = VM_EXITCODE_SPINUP_AP;
                        vmexit->u.spinup_ap.vcpu = dest;
                        vmexit->u.spinup_ap.rip = vec << PAGE_SHIFT;

                        return (0);
                }
        }

This is the only MMIO handler to operate in this manner, and to have it modifying exitinfo directly is somewhat unexpected compared to other cases. Given the restructuring necessary for #12989, it would be nice to create a more formal mechanism for the vlapic to request a SPINUP_AP exit, rather than modifying exitinfo directly.


Related issues

Has duplicate illumos gate - Bug #12960: bhyve needs better SIPI notificationClosedPatrick Mooney

Actions
Actions #1

Updated by Electric Monk 11 months ago

  • Gerrit CR set to 815
Actions #2

Updated by Patrick Mooney 10 months ago

Testing notes for this (and #12990) are written up in #12989.

Actions #3

Updated by Electric Monk 10 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit e0c0d44e917080841514d0dd031a696c74e8c435

commit  e0c0d44e917080841514d0dd031a696c74e8c435
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2020-08-20T19:19:51.000Z

    12989 improve interface boundary for bhyve MMIO
    12990 improve interface boundary for bhyve ins/outs
    12991 bhyve vlapic should SIPI more carefully
    Reviewed by: Mike Zeller <mike.zeller@joyent.com>
    Reviewed by: Joshua M. Clulow <josh@sysmgr.org>
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Approved by: Gordon Ross <gordon.w.ross@gmail.com>

Actions #4

Updated by Patrick Mooney 9 months ago

  • Has duplicate Bug #12960: bhyve needs better SIPI notification added
Actions

Also available in: Atom PDF