Feature #14090
closed
ld(1) could use a normal allocator
100%
Description
At the minute, ld(1) uses a custom, mmap-based allocator and in the usual case frees no memory (apparently for performance).
This has proven to be a major disadvantage in the face of large-scale metaprogramming in the manner now common to C++ and Rust which may create vast amounts of input data, the vast majority of which we will discard.
The first step of the journey to make this better is to use a regular memory allocator, so that we can see what's going on and begin shaking out the low-hanging fruit, and hunting for the biggest wins.
We should make ld(1) use libumem, and libld no longer contain a custom allocator, falling back on the allocators of its consumers (such that ld(1) now uses umem, but ld.so.1 still uses its own custom libc-free malloc).
Related issues
Updated by Rich Lowe 8 months ago
testing:
Built illumos, installed the result and built illumos again and used the system because of the possible impact on ld.so
Checked we get the ld.so internal malloc
Checked a running binary on the new system doesn't contain umem in ld.so's linkmap.
Checked, in combination with jclulow, that we can extract useful umem debugging information from the link editor now.
Checked, in combination with jclulow, that allocator metadata doesn't (temporarily) make our problems worse.
Updated by Electric Monk 8 months ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
git commit fa4a3e77edc40df6d92e8da6fc4961d275e9896d
commit fa4a3e77edc40df6d92e8da6fc4961d275e9896d
Author: Richard Lowe <richlowe@richlowe.net>
Date: 2021-09-29T18:57:11.000Z
14090 ld(1) could use a normal allocator
Reviewed by: Gordon Ross <Gordon.W.Ross@gmail.com>
Reviewed by: Jason King <jason.brian.king+illumos@gmail.com>
Approved by: Robert Mustacchi <rm@fingolfin.org>
Updated by Marcel Telka 8 months ago
- Related to Bug #14127: ld(1) can double free when cleaning up added
Updated by Electric Monk about 1 month ago
git commit fb12490ab4d1e87e7a63e457dd4fba1ea34c402a
commit fb12490ab4d1e87e7a63e457dd4fba1ea34c402a
Author: Richard Lowe <richlowe@richlowe.net>
Date: 2022-04-21T20:37:47.000Z
14090 ld(1) could use a normal allocator
Reviewed by: Gordon Ross <Gordon.W.Ross@gmail.com>
Reviewed by: Jason King <jason.brian.king+illumos@gmail.com>
Reviewed by: Michael van der Westhuizen <r1mikey@gmail.com>
Approved by: Dan McDonald <danmcd@joyent.com>