Bug #16572


loader should be willing to put modules above 4GB if necessary

Added by Joshua M. Clulow 25 days ago. Updated 1 day ago.

Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:
External Bug:


Some newer systems put the vast majority of physical memory at an address above the 4GB boundary; e.g., on a System76 Thelio system we recently observed the following chunks of available memory:

TYPE                PHYSICAL        NPAGES
ConventionalMemory  000000100000    0001fb2b        (507MB)

   (and then nothing very big until...)

ConventionalMemory  000100000000    00fbe000        (62.9GB)

Sometimes it is useful to be able to boot a gzip-compressed ramdisk that unpacks into a large (upwards of 500-600MB, say) UFS file system, especially on modern systems with a vast quantity of DRAM.

On an EFI system, we use efi_loadaddr() to allocate memory, which itself uses the boot services AllocatePages() routine:

154  /*
155   * Allocate pages for data to be loaded. As we can not expect AllocateAddress
156   * to succeed, we allocate using AllocateMaxAddress from 4GB limit.
157   * 4GB limit is because reportedly some 64bit systems are reported to have
158   * issues with memory above 4GB. It should be quite enough anyhow.
159   * Note: AllocateMaxAddress will only make sure we are below the specified
160   * address, we can not make any assumptions about actual location or
161   * about the order of the allocated blocks.
162   */
163  vm_offset_t
164  efi_loadaddr(uint_t type, void *data, vm_offset_t addr)
165  {
166      EFI_PHYSICAL_ADDRESS paddr;
167      struct stat st;
168      size_t size;
169      uint64_t pages;
170      EFI_STATUS status;
172      if (addr == 0)
173          return (addr);    /* nothing to do */
175      if (type == LOAD_ELF)
176          return (0);    /* not supported */
178      if (type == LOAD_MEM)
179          size = *(size_t *)data;
180      else {
181          stat(data, &st);
182          size = st.st_size;
183      }
185      /* AllocatePages can not allocate 0 pages. */
186      if (size == 0)
187          return (addr);
189      pages = EFI_SIZE_TO_PAGES(size);
190      /* 4GB upper limit */
191      paddr = UINT32_MAX;
193      status = BS->AllocatePages(AllocateMaxAddress, EfiLoaderData,
194          pages, &paddr);
196      if (EFI_ERROR(status)) {
197          printf("failed to allocate %zu bytes for staging area: %lu\n",
198              size, DECODE_ERROR(status));
199          return (0);
200      }
202      return (paddr);
203  }

Note on 190, we constrain the allocation so that it entirely fits under the 4GB boundary. This means we cannot use a large ramdisk on some EFI systems, even though 4GB "should be quite enough". As per the comment, "reportedly some 64bit systems are reported to have issues with memory above 4GB", but it's not really clear what those issues are. Certainly in this case, on this system, we should at least try to allocate above 4GB, because the alternative is simply not booting at all.

To avoid exciting new issues on machines that otherwise would have booted correctly with this restriction in place, we should probably try one allocation under 4GB, and iff that fails, try again without a constraint to see if we can get memory at a higher address, on the assumption that in 2024, a system which puts most of its memory above the 4GB boundary is probably otherwise fine with high allocations -- or, at least, that it's better to try than not to boot!

Related issues

Related to illumos gate - Bug #14060: loader.efi: multiboot2_exec() should fall back to use module load addressClosedToomas Soome

Actions #1

Updated by Joshua M. Clulow 24 days ago

  • Assignee set to Joshua M. Clulow
Actions #2

Updated by Joshua M. Clulow 1 day ago

  • Related to Bug #14060: loader.efi: multiboot2_exec() should fall back to use module load address added
Actions #3

Updated by Joshua M. Clulow 1 day ago

I have uncovered some more malaise: multiboot2 only supports 32-bit addresses for modules! Fortunately, we're not trying to boot arbitrary multiboot2 kernels, just ours, so we can create a new OS-specific tag type that allows for 64-bit module addresses and use that new wider tag to express modules that can't be loaded under 4GB.

We'll have to be careful in dboot_startkern as well: it won't be able to check the hash on a module that's loaded above 4GB, as it's built to run in 32-bit identity mapped mode. I don't believe we otherwise need to touch the contents of the high-mapped modules, though, until after we bounce up into the 64-bit kernel.


Also available in: Atom PDF