Project

General

Profile

Actions

Bug #13381

closed

support TSC offsets on AMD bhyve

Added by Patrick Mooney 12 months ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

The logic for applying TSC offsets to bhyve guests on AMD CPUs was never wired up. This is necessary for a few reasons:
1. If host CPU TSCs happen to be skewed relative to each other, that offset needs to be applied when migrating between them
2. To support guest writes to the TSC MSR, applying its own relative offset
3. Whenever migration support is added, preserving the TSC offset between two different machines

Actions #1

Updated by Patrick Mooney 12 months ago

  • Status changed from New to In Progress
Actions #2

Updated by Electric Monk 12 months ago

  • Gerrit CR set to 1110
Actions #3

Updated by Patrick Mooney 10 months ago

Test notes from Mike Zeller:

I did some additional testing with the rebased branch.
I have two VMs running on an AMD TR box. One is alpine linux which doesn't display any offset and the other is running haiku and the offset is present:

vmm_list::walk list | ::print vmm_softc_t vmm_vm | ::print -t struct vm vcpu | ::array struct vcpu 2 | ::print -t struct vcpu tsc_offset
uint64_t tsc_offset = 0
uint64_t tsc_offset = 0
uint64_t tsc_offset = 0xffffffe46b352776
uint64_t tsc_offset = 0xffffffe46b350bb4
Actions #4

Updated by Patrick Mooney 10 months ago

One avenue I used for testing this was a custom illumos guest which was configured to unconditionally reset its TSCs (ala the sandy/ivy bridge errata) during boot, and then measure the induced skew between them. That skew could then be compared against the per-vcpu offsets in the bhyve host. When running the test, the offsets appeared to be appropriately close.

Actions #5

Updated by Patrick Mooney 10 months ago

I also ran this through the typical guest OS smoke test on AMD and intel.

Actions #6

Updated by Electric Monk 10 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 9250eb132f1fc3950c838b7a5746f1bcdac7eba1

commit  9250eb132f1fc3950c838b7a5746f1bcdac7eba1
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2021-02-09T04:09:15.000Z

    13375 bhyve should expose invariant TSC
    13381 support TSC offsets on AMD bhyve
    13382 consolidate MSR emulation in bhyve
    Reviewed by: Andy Fiddaman <andy@omnios.org>
    Reviewed by: Jason King <jason.king@joyent.com>
    Reviewed by: Mike Zeller <mike.zeller@joyent.com>
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Approved by: Dan McDonald <danmcd@joyent.com>

Actions

Also available in: Atom PDF