Project

General

Profile

Actions

Feature #13840

closed

bhyve hostbridge should feign PAM0 emulation

Added by Patrick Mooney 16 days ago. Updated 10 days ago.

Status:
Closed
Priority:
Normal
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

While testing the initial patch for #13835, we found that old CSM bootroms succumbed to the issue which prompted that memory zeroing in the first place: broken behavior when an instance reboots. After adding some debug logic to zero a selected range during boot, I narrowed it down to 0xf0000-0xfffff (physical) which, if zeroed during reboot, would allow the CSM bootrom to make process like normal. Some research lead me to Intel docs which describe that as "Programmable Attribute Map (PAM)":

PAM is a legacy BIOS ROM area in MMIO. It is overlaid with DRAM and used as a faster ROM storage area. It has a fixed base address (000C_0000h) and fixed size of 256 KB. The 13 sections from 768 KB to 1 MB comprise what is also known as the PAM Memory Area. Each section has Read enable and Write enable attributes.

(From 10th Generation IntelĀ® Processor Families)

The i440FX datasheet had more detail about PAM, and qemu appears to emulate the changes in behavior for those address ranges when their configuration registers are modified. The registers bear the default value of zero which implies that accesses would bypass DRAM:

Disabled. DRAM is disabled and all accesses are directed to PCI. PMC does not respond as a PCI target for any read or write access to this area.

The hostbridge emulation in bhyve does none of this, so it's not surprising that the bootrom acted strangely when those physical addresses were backed with DRAM, persisting values across reboot.

A short term solution to this problem would be to have the hostbridge emulation zero that PAM0 (0xf0000-0xfffff) region of physical guest memory on boot, allowing #13835 to proceed while preserving CSM bootrom functionality with C-bhyve.


Related issues

Related to illumos gate - Feature #13835: bhyve should not zero lowmem on rebootClosedPatrick Mooney

Actions
Actions #1

Updated by Patrick Mooney 16 days ago

Quicky hack I used for narrowing down that memory region:

diff --git a/usr/src/cmd/bhyve/bhyverun.c b/usr/src/cmd/bhyve/bhyverun.c
index c777529858..841cf2ed70 100644
--- a/usr/src/cmd/bhyve/bhyverun.c
+++ b/usr/src/cmd/bhyve/bhyverun.c
@@ -1365,6 +1365,39 @@ set_defaults(void)
        set_config_bool("x86.strictmsr", true);
 }

+static void
+prezero_mem(struct vmctx *ctx, const char *opt)
+{
+       char *buf, *save;
+       const char *start, *len;
+       unsigned long nstart = ULONG_MAX, nlen = ULONG_MAX;
+
+
+       buf = strdup(opt);
+       start = strtok_r(buf, ":", &save);
+       len = strtok_r(NULL, ":", &save);
+
+       if (len != NULL) {
+               nstart = strtoul(start, NULL, 16);
+               nlen = strtoul(len, NULL, 16);
+       }
+       if (nstart == ULONG_MAX || nlen == ULONG_MAX) {
+               fprintf(stderr, "Could not parse memory pre-zero range: '%s'\n",
+                  opt);
+               exit(1);
+       }
+
+       void *zero_region = vm_map_gpa(ctx, nstart, nlen);
+       if (zero_region == NULL) {
+               fprintf(stderr, "Invalid pre-zeroing region: [%lx, %lx)\n",
+                   nstart, nstart + nlen );
+               exit(1);
+       }
+
+       bzero(zero_region, nlen);
+       fprintf(stderr, "Pre-zeroed [%lx, %lx)\n", nstart, nstart + nlen);
+}
+
 int
 main(int argc, char *argv[])
 {
@@ -1572,6 +1605,13 @@ main(int argc, char *argv[])
                exit(4);
        }

+#ifndef        __FreeBSD__
+       value = get_config_value("memory.prezero");
+       if (value != NULL) {
+               prezero_mem(ctx, value);
+       }
+#endif
+
        error = init_msr();
        if (error) {
                fprintf(stderr, "init_msr error %d", error);

Actions #2

Updated by Electric Monk 16 days ago

  • Gerrit CR set to 1523
Actions #3

Updated by Patrick Mooney 11 days ago

  • Related to Feature #13835: bhyve should not zero lowmem on reboot added
Actions #4

Updated by Electric Monk 10 days ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 9e6c6f2f3ab9c8f1f042b97c0432e8ccc9650309

commit  9e6c6f2f3ab9c8f1f042b97c0432e8ccc9650309
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2021-06-03T01:52:27.000Z

    13835 bhyve should not zero lowmem on reboot
    13840 bhyve hostbridge should feign PAM0 emulation
    Reviewed by: Andy Fiddaman <andy@omnios.org>
    Reviewed by: Dan Cross <cross@oxidecomputer.com>
    Approved by: Robert Mustacchi <rm@fingolfin.org>

Actions

Also available in: Atom PDF