Project

General

Profile

Bug #13016

Updated by Patrick Mooney over 1 year ago

While testing a new build of the UEFI ROM on an older Intel machine, andyf was observing bhyve exit on EFAULT: 
 <pre> 
 vm_run error -1, errno 14 
 </pre> 

 After some dtracing, this was found to be emitted by the instruction emulation logic while it attempted to fetch a MMIO instruction directly from the bootrom: 
 <pre> 
 1    68812               vm_copy_setup:return                  14 
 1    68896       vmm_fetch_instruction:return                  14 
 1    67872         vm_handle_inst_emul:return                  14 
 </pre> 

 Further digging confirmed that @%rip@ pointed to a @mov@ instruction which was attempting to read from the APIC. 
 <pre> 
 0xfffcdd2e:                       movl     (%rbx),%esi 
 %rbx = 0x00000000fee000f0 
 </pre> 

 This is a little bit strange.    Surely the uefi bootrom, old and new, has been accessing MMIO resources during bootup.    It does appear as if the ROM relocates its text into "normal" system DRAM before getting into the bulk of its work, so perhaps the new build has changed the order of certain component initialization such that it now accesses these APIC resources before the relocation occurs.  

 With that concern out of the way, let's turn our eye to _why_ it cannot fetch the instruction for decoding.    Further tracing revealed that @vm_gpa_hold@ was failing during the instruction fetch.    Looking at its source, the reason becomes apparent: 
 <pre> 
 ... 

         count = 0; 
         for (i = 0; i < VM_MAX_MEMMAPS; i++) { 
                 mm = &vm->mem_maps[i]; 
                 if (sysmem_mapping(vm, mm) && gpa >= mm->gpa && 
                     gpa < mm->gpa + mm->len) { 
                         count = vm_fault_quick_hold_pages(&vm->vmspace->vm_map, 
                             trunc_page(gpa), PAGE_SIZE, reqprot, &m, 1); 
                         break; 
                 } 
         } 
 ... 
 </pre> 

 The bootrom, being a somewhat special resource, is not considered "sysmem" (unlike the normal physical memory presented to the guest).    For that reason, bhyve is not allowing a hold be placed on that GPA before the copy is initiated.    Looking at the bhyve memory map logic, there's no clear reason why this is enforced.    We should be able to lift this specific limitation. 

 It should be noted that this failure was found only on Intel hardware without APICv support.    With APICv, the accesses performed by the bootrom prior to relocation were able to be accelerated, so no instruction emulation was required.    I reproduced this on an APICv-capable machine by disabling APICv via @vmx_capabilities*. 

Back