Bug #12998

OpenBSD needs DE_CFG MSR on AMD bhyve

Added by Patrick Mooney 10 days ago. Updated 7 days ago.

Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


When attempting to boot openbsd 6.7 under bhyve on an AMD (Rome) machine, it attempts to unconditionally write to the DE_CFG (0xc0011029) MSR. When this is done from the installer, the system simply panics on the #GP and halts. For an already-installed VM, it'll drop into the debugger on that instruction:

kernel: protection fault trap, code=0
Stopped at      identifycpu+0xa5c:      wrmsr
ddb> show registers
rdi               0xffffffff81f12035    cpu_vendor+0xd
rsi               0xffffffff81c5dacb    cmd0646_9_tim_udma+0x15c35
rbp               0xffffffff822618c0    end+0x2618c0
rbx                                0
rdx                                0
rcx                       0xc0011029
rax                              0x3
r8                0xffffffff82261800    end+0x261800
r9                0xffffffff82261667    end+0x261667
r10               0x633b42a234a4dd88
r11               0x2ae3b7a13024bb27
r12                              0x1
r13               0xffffffff81ef7030    cpu_cd
r14               0xffffffff81ef5ff0    cpu_info_full_primary+0x1ff0
r15                       0xa69c1f70
rip               0xffffffff812906cc    identifycpu+0xa5c
cs                               0x8
rflags                       0x10206    __ALIGN_SIZE+0xf206
rsp               0xffffffff82261850    end+0x261850
ss                              0x10
identifycpu+0xa5c:      wrmsr

When the MSR is emulated as a no-op, the VM is able to boot and run without any issues.

Related issues

Related to illumos gate - Bug #13002: bhyve should expose lfence-serialize stateNew




Updated by Patrick Mooney 10 days ago

Scraping some public AMD docs for how they've defined certain bits in this MSR for different families.

Family 10h:
0 - Workaround for errata 721 on family 10h
1 - Make LFENCE serializing - "All AMD family 10h/12h/14h/15h/16h/17h processors support this MSR. LFENCE support is indicated by CPUID function1 EDX bit 26, SSE2. AMD family 0Fh/11h processors support LFENCE as serializing always but do not support this MSR. AMD plans support for this MSR and access to this bit for all future processors."

Family 12h:
23 - ClflushSerialize. Read-write. 1=CLFLUSH is a serializing instruction. 0=CLFLUSH is not a serializing instruction.

Family 15h:
2:7 - ConvertPrefetchToNop. Read-write. Reset: 0. 1=The specified PREFETCH instruction is converted into a NOP. Each of these bits apply to a different instruction as follows: ...
10 - ResyncPredSingleDispDis. Read-write. Reset: 0. BIOS: IF (PROC>=OR_C0) THEN 0 ELSE 1 ENDIF1. 1=Disables the single dispatch of loads that hit in the resync predictor.

Family 17h:
13 - Workaround for errata 1021


Updated by Patrick Mooney 10 days ago

  • Subject changed from openbsd needs DE_CFG MSR on AMD bhyve to OpenBSD needs DE_CFG MSR on AMD bhyve

Updated by Electric Monk 10 days ago

  • Gerrit CR set to 822

Updated by Patrick Mooney 10 days ago

With the proposed fix, an OpenBSD 6.7 guest is able to start on bhyve without panicking on the DE_CFG MSR access. A centos guest, which previously would access that MSR, but tolerate the failure, still boots fine as well.


Updated by Patrick Mooney 8 days ago

  • Related to Bug #13002: bhyve should expose lfence-serialize state added

Updated by Electric Monk 7 days ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 425a83377a83720d1e9907ec3641a499a5eed18f

commit  425a83377a83720d1e9907ec3641a499a5eed18f
Author: Patrick Mooney <>
Date:   2020-08-01T18:53:27.000Z

    12998 OpenBSD needs DE_CFG MSR on AMD bhyve
    Reviewed by: Andy Fiddaman <>
    Reviewed by: Mike Zeller <>
    Reviewed by: Robert Mustacchi <>
    Approved by: Dan McDonald <>

Also available in: Atom PDF