Bug #13686


bhyve should emulate mov-to-%cr

Added by Patrick Mooney about 1 year ago. Updated 8 months ago.

Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


Jonathan Perkin reported a machine with an AMD N36L CPU failing to boot a guest after updating to a platform featuring #13256.

The specific error, reported on SmartOS PI joyent_20210311T001742Z is:

[2021-03-25T10:47:56.023760000Z]  INFO: zoneadmd/79101 on (stream=stderr)
     Failed to emulate instruction sequence [0f, 22, c0, 66, ea, 5c, ff, ff, ff, 10, 00, b8, 40, 06, 00] @ rip = ff51

Dumping that to a file to see what the disassembly is resulted in:

   00000000 <.data>:
      0:   0f 22 c0                mov    %eax,%cr0
      3:   66 ea 5c ff ff ff       ljmpw  $0xffff,$0xff5c
      9:   10 00                   adc    %al,(%eax)
      b:   b8                      .byte 0xb8
      c:   40                      inc    %eax
      d:   06                      push   %es

Since mov-to-%cr0 is normally covered by the SVM instruction decode assist, I inquired about the relevant CPUID reading from that node:

cpi_extd[a].cp_edx = 0xf

Without bit 7 set, it's clear that this CPU lacks the decode assists. With the advent of %cr0 shadowing, bhyve will take exits for guest actions against %cr0 which would have previously been allowed to occur unimpeded. In order to continue supporting those older CPUs, we'll need to add instruction emulation for the mov-to-%cr instructions.

Related issues

Related to illumos gate - Bug #13256: bhyve should shadow %cr0 on AMDClosedPatrick Mooney

Actions #1

Updated by Patrick Mooney about 1 year ago

  • Related to Bug #13256: bhyve should shadow %cr0 on AMD added
Actions #2

Updated by Patrick Mooney 9 months ago

jperkin tested the proposed fix on his afflicted machine with positive results:

This looks good to me, thanks! I tested the patch applied to latest SmartOS:

  $ uname -v

  $ vmadm list type=BHYV
  UUID                                  TYPE  RAM      STATE             ALIAS
  8a8fb5af-31f2-4ce9-88b4-fac8b86419bc  BHYV  1024     running           netbsd9

Previously any bhyve VMs would not boot at all.

Actions #3

Updated by Patrick Mooney 8 months ago

Without an AMD machine of the same vintage to call my own, I was not able to test the specific issue myself (leaving that, instead, to jperkin), but I did run through the typical battery of guests on AMD and Intel hardware to check for regressions. They all booted and ran normally.

Actions #4

Updated by Electric Monk 8 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 418ddc2681444d3bbf1ac20f216a4e011cf6c630

commit  418ddc2681444d3bbf1ac20f216a4e011cf6c630
Author: Patrick Mooney <>
Date:   2021-09-10T17:12:07.000Z

    13686 bhyve should emulate mov-to-%cr
    Reviewed by: Jonathan Perkin <>
    Reviewed by: Andy Fiddaman <>
    Approved by: Robert Mustacchi <>


Also available in: Atom PDF