Project

General

Profile

Bug #12998

OpenBSD needs DE_CFG MSR on AMD bhyve

Added by Patrick Mooney 10 days ago. Updated 7 days ago.

Status:
Closed
Priority:
Normal
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
bhyve
Gerrit CR:

Description

When attempting to boot openbsd 6.7 under bhyve on an AMD (Rome) machine, it attempts to unconditionally write to the DE_CFG (0xc0011029) MSR. When this is done from the installer, the system simply panics on the #GP and halts. For an already-installed VM, it'll drop into the debugger on that instruction:

kernel: protection fault trap, code=0
Stopped at      identifycpu+0xa5c:      wrmsr
...
ddb> show registers
rdi               0xffffffff81f12035    cpu_vendor+0xd
rsi               0xffffffff81c5dacb    cmd0646_9_tim_udma+0x15c35
rbp               0xffffffff822618c0    end+0x2618c0
rbx                                0
rdx                                0
rcx                       0xc0011029
rax                              0x3
r8                0xffffffff82261800    end+0x261800
r9                0xffffffff82261667    end+0x261667
r10               0x633b42a234a4dd88
r11               0x2ae3b7a13024bb27
r12                              0x1
r13               0xffffffff81ef7030    cpu_cd
r14               0xffffffff81ef5ff0    cpu_info_full_primary+0x1ff0
r15                       0xa69c1f70
rip               0xffffffff812906cc    identifycpu+0xa5c
cs                               0x8
rflags                       0x10206    __ALIGN_SIZE+0xf206
rsp               0xffffffff82261850    end+0x261850
ss                              0x10
identifycpu+0xa5c:      wrmsr

When the MSR is emulated as a no-op, the VM is able to boot and run without any issues.


Related issues

Related to illumos gate - Bug #13002: bhyve should expose lfence-serialize stateNew

Actions

History

#1

Updated by Patrick Mooney 10 days ago

Scraping some public AMD docs for how they've defined certain bits in this MSR for different families.

Family 10h:
0 - Workaround for errata 721 on family 10h
1 - Make LFENCE serializing - "All AMD family 10h/12h/14h/15h/16h/17h processors support this MSR. LFENCE support is indicated by CPUID function1 EDX bit 26, SSE2. AMD family 0Fh/11h processors support LFENCE as serializing always but do not support this MSR. AMD plans support for this MSR and access to this bit for all future processors."

Family 12h:
23 - ClflushSerialize. Read-write. 1=CLFLUSH is a serializing instruction. 0=CLFLUSH is not a serializing instruction.

Family 15h:
2:7 - ConvertPrefetchToNop. Read-write. Reset: 0. 1=The specified PREFETCH instruction is converted into a NOP. Each of these bits apply to a different instruction as follows: ...
10 - ResyncPredSingleDispDis. Read-write. Reset: 0. BIOS: IF (PROC>=OR_C0) THEN 0 ELSE 1 ENDIF1. 1=Disables the single dispatch of loads that hit in the resync predictor.

Family 17h:
13 - Workaround for errata 1021

#2

Updated by Patrick Mooney 10 days ago

  • Subject changed from openbsd needs DE_CFG MSR on AMD bhyve to OpenBSD needs DE_CFG MSR on AMD bhyve
#3

Updated by Electric Monk 10 days ago

  • Gerrit CR set to 822
#4

Updated by Patrick Mooney 10 days ago

With the proposed fix, an OpenBSD 6.7 guest is able to start on bhyve without panicking on the DE_CFG MSR access. A centos guest, which previously would access that MSR, but tolerate the failure, still boots fine as well.

#5

Updated by Patrick Mooney 8 days ago

  • Related to Bug #13002: bhyve should expose lfence-serialize state added
#6

Updated by Electric Monk 7 days ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 425a83377a83720d1e9907ec3641a499a5eed18f

commit  425a83377a83720d1e9907ec3641a499a5eed18f
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2020-08-01T18:53:27.000Z

    12998 OpenBSD needs DE_CFG MSR on AMD bhyve
    Reviewed by: Andy Fiddaman <andy@omniosce.org>
    Reviewed by: Mike Zeller <mike.zeller@joyent.com>
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF