ld(1) could use a normal allocator
At the minute, ld(1) uses a custom, mmap-based allocator and in the usual case frees no memory (apparently for performance).
This has proven to be a major disadvantage in the face of large-scale metaprogramming in the manner now common to C++ and Rust which may create vast amounts of input data, the vast majority of which we will discard.
The first step of the journey to make this better is to use a regular memory allocator, so that we can see what's going on and begin shaking out the low-hanging fruit, and hunting for the biggest wins.
We should make ld(1) use libumem, and libld no longer contain a custom allocator, falling back on the allocators of its consumers (such that ld(1) now uses umem, but ld.so.1 still uses its own custom libc-free malloc).
Built illumos, installed the result and built illumos again and used the system because of the possible impact on ld.so
Checked we get the ld.so internal malloc
Checked a running binary on the new system doesn't contain umem in ld.so's linkmap.
Checked, in combination with jclulow, that we can extract useful umem debugging information from the link editor now.
Checked, in combination with jclulow, that allocator metadata doesn't (temporarily) make our problems worse.
Updated by Electric Monk 8 months ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
commit fa4a3e77edc40df6d92e8da6fc4961d275e9896d Author: Richard Lowe <firstname.lastname@example.org> Date: 2021-09-29T18:57:11.000Z 14090 ld(1) could use a normal allocator Reviewed by: Gordon Ross <Gordon.W.Ross@gmail.com> Reviewed by: Jason King <email@example.com> Approved by: Robert Mustacchi <firstname.lastname@example.org>
Updated by Electric Monk about 1 month ago
commit fb12490ab4d1e87e7a63e457dd4fba1ea34c402a Author: Richard Lowe <email@example.com> Date: 2022-04-21T20:37:47.000Z 14090 ld(1) could use a normal allocator Reviewed by: Gordon Ross <Gordon.W.Ross@gmail.com> Reviewed by: Jason King <firstname.lastname@example.org> Reviewed by: Michael van der Westhuizen <email@example.com> Approved by: Dan McDonald <firstname.lastname@example.org>