Feature #13840
closed
bhyve hostbridge should feign PAM0 emulation
100%
Description
While testing the initial patch for #13835, we found that old CSM bootroms succumbed to the issue which prompted that memory zeroing in the first place: broken behavior when an instance reboots. After adding some debug logic to zero a selected range during boot, I narrowed it down to 0xf0000-0xfffff (physical) which, if zeroed during reboot, would allow the CSM bootrom to make process like normal. Some research lead me to Intel docs which describe that as "Programmable Attribute Map (PAM)":
PAM is a legacy BIOS ROM area in MMIO. It is overlaid with DRAM and used as a faster ROM storage area. It has a fixed base address (
000C_0000h) and fixed size of 256 KB. The 13 sections from 768 KB to 1 MB comprise what is also known as the PAM Memory Area. Each section has Read enable and Write enable attributes.
(From 10th Generation Intel® Processor Families)
The i440FX datasheet had more detail about PAM, and qemu appears to emulate the changes in behavior for those address ranges when their configuration registers are modified. The registers bear the default value of zero which implies that accesses would bypass DRAM:
Disabled. DRAM is disabled and all accesses are directed to PCI. PMC does not respond as a PCI target for any read or write access to this area.
The hostbridge emulation in bhyve does none of this, so it's not surprising that the bootrom acted strangely when those physical addresses were backed with DRAM, persisting values across reboot.
A short term solution to this problem would be to have the hostbridge emulation zero that PAM0 (0xf0000-0xfffff) region of physical guest memory on boot, allowing #13835 to proceed while preserving CSM bootrom functionality with C-bhyve.
Related issues
Updated by Patrick Mooney about 1 year ago
Quicky hack I used for narrowing down that memory region:
diff --git a/usr/src/cmd/bhyve/bhyverun.c b/usr/src/cmd/bhyve/bhyverun.c
index c777529858..841cf2ed70 100644
--- a/usr/src/cmd/bhyve/bhyverun.c
+++ b/usr/src/cmd/bhyve/bhyverun.c
@@ -1365,6 +1365,39 @@ set_defaults(void)
set_config_bool("x86.strictmsr", true);
}
+static void
+prezero_mem(struct vmctx *ctx, const char *opt)
+{
+ char *buf, *save;
+ const char *start, *len;
+ unsigned long nstart = ULONG_MAX, nlen = ULONG_MAX;
+
+
+ buf = strdup(opt);
+ start = strtok_r(buf, ":", &save);
+ len = strtok_r(NULL, ":", &save);
+
+ if (len != NULL) {
+ nstart = strtoul(start, NULL, 16);
+ nlen = strtoul(len, NULL, 16);
+ }
+ if (nstart == ULONG_MAX || nlen == ULONG_MAX) {
+ fprintf(stderr, "Could not parse memory pre-zero range: '%s'\n",
+ opt);
+ exit(1);
+ }
+
+ void *zero_region = vm_map_gpa(ctx, nstart, nlen);
+ if (zero_region == NULL) {
+ fprintf(stderr, "Invalid pre-zeroing region: [%lx, %lx)\n",
+ nstart, nstart + nlen );
+ exit(1);
+ }
+
+ bzero(zero_region, nlen);
+ fprintf(stderr, "Pre-zeroed [%lx, %lx)\n", nstart, nstart + nlen);
+}
+
int
main(int argc, char *argv[])
{
@@ -1572,6 +1605,13 @@ main(int argc, char *argv[])
exit(4);
}
+#ifndef __FreeBSD__
+ value = get_config_value("memory.prezero");
+ if (value != NULL) {
+ prezero_mem(ctx, value);
+ }
+#endif
+
error = init_msr();
if (error) {
fprintf(stderr, "init_msr error %d", error);
Updated by Patrick Mooney 12 months ago
- Related to Feature #13835: bhyve should not zero lowmem on reboot added
Updated by Electric Monk 12 months ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
git commit 9e6c6f2f3ab9c8f1f042b97c0432e8ccc9650309
commit 9e6c6f2f3ab9c8f1f042b97c0432e8ccc9650309
Author: Patrick Mooney <pmooney@pfmooney.com>
Date: 2021-06-03T01:52:27.000Z
13835 bhyve should not zero lowmem on reboot
13840 bhyve hostbridge should feign PAM0 emulation
Reviewed by: Andy Fiddaman <andy@omnios.org>
Reviewed by: Dan Cross <cross@oxidecomputer.com>
Approved by: Robert Mustacchi <rm@fingolfin.org>