Project

General

Profile

Actions
Copy link

Bug #13381

closed

support TSC offsets on AMD bhyve

Added by Patrick Mooney over 2 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Patrick Mooney
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
1110
External Bug:

Description

The logic for applying TSC offsets to bhyve guests on AMD CPUs was never wired up. This is necessary for a few reasons:
1. If host CPU TSCs happen to be skewed relative to each other, that offset needs to be applied when migrating between them
2. To support guest writes to the TSC MSR, applying its own relative offset
3. Whenever migration support is added, preserving the TSC offset between two different machines

Actions
Copy link
 #1

Updated by Patrick Mooney over 2 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #2

Updated by Electric Monk over 2 years ago

  • Gerrit CR set to 1110
Actions
Copy link
 #3

Updated by Patrick Mooney over 2 years ago

Test notes from Mike Zeller:

I did some additional testing with the rebased branch.
I have two VMs running on an AMD TR box. One is alpine linux which doesn't display any offset and the other is running haiku and the offset is present:

vmm_list::walk list | ::print vmm_softc_t vmm_vm | ::print -t struct vm vcpu | ::array struct vcpu 2 | ::print -t struct vcpu tsc_offset
uint64_t tsc_offset = 0
uint64_t tsc_offset = 0
uint64_t tsc_offset = 0xffffffe46b352776
uint64_t tsc_offset = 0xffffffe46b350bb4
Actions
Copy link
 #4

Updated by Patrick Mooney over 2 years ago

One avenue I used for testing this was a custom illumos guest which was configured to unconditionally reset its TSCs (ala the sandy/ivy bridge errata) during boot, and then measure the induced skew between them. That skew could then be compared against the per-vcpu offsets in the bhyve host. When running the test, the offsets appeared to be appropriately close.

Actions
Copy link
 #5

Updated by Patrick Mooney over 2 years ago

I also ran this through the typical guest OS smoke test on AMD and intel.

Actions
Copy link
 #6

Updated by Electric Monk over 2 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 9250eb132f1fc3950c838b7a5746f1bcdac7eba1

commit  9250eb132f1fc3950c838b7a5746f1bcdac7eba1
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2021-02-09T04:09:15.000Z

    13375 bhyve should expose invariant TSC
    13381 support TSC offsets on AMD bhyve
    13382 consolidate MSR emulation in bhyve
    Reviewed by: Andy Fiddaman <andy@omnios.org>
    Reviewed by: Jason King <jason.king@joyent.com>
    Reviewed by: Mike Zeller <mike.zeller@joyent.com>
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Approved by: Dan McDonald <danmcd@joyent.com>

Actions
Copy link

Also available in: Atom PDF