Project

General

Profile

Bug #13252

ctf_update()/ctf_dwarf_convert_function() leak memory

Added by Andy Fiddaman about 1 month ago. Updated 15 days ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
tools - gate/build tools
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

While troubleshooting CTF conversion for large objects, I hit a problem with a shared library which has a large number of CUs. Having fixed the original memory utilisation via issue 13247, it still failed with a mmap() failure.

% ctfconvert -o /dev/null ~/ctf/pkgsrc.libicudata.so.67.1
ctfconvert: CTF conversion failed: Failed to mmap a needed data section

% truss -c  ctfconvert -o /dev/null ~/ctf/pkgsrc.libicudata.so.67.1
ctfconvert: CTF conversion failed: Failed to mmap a needed data section

syscall               seconds   calls  errors
mmap                   17.593   39667
munmap                  2.658   33698
mmap64                 10.247   34161       1

I temporarily switched in libumem and found a couple of leaks:

Total    5030 buffers, 45176472 bytes

They came from two places - ctf_update() and ctf_dwarf_convert_function()
This is just from converting the relatively small nginx binary.

Having fixed these, it is now possible to convert the original libicudata object, with much lower memory usage - prior to this and 13247, it was exceeding the 32-bit memory space.

% prstat -c -p `pgrep -n ctfconvert` 1
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
  5369 af        111M   82M cpu21    1    0   0:00:20 0.8% ctfconvert/4
  5369 af         69M   39M cpu25    1    0   0:00:23 0.8% ctfconvert/4
  5369 af        227M  197M cpu29    1    0   0:00:26 0.8% ctfconvert/1
  5369 af        222M  193M cpu29    1    0   0:00:28 0.9% ctfconvert/1
  5369 af        115M   85M cpu7     1    0   0:00:31 1.1% ctfconvert/4
  5369 af         90M   61M cpu29    1    0   0:00:33 1.0% ctfconvert/1
  5369 af        228M  199M cpu15    1    0   0:00:36 1.1% ctfconvert/4
  5369 af        136M  106M cpu25    1    0   0:00:38 1.3% ctfconvert/4
  5369 af         68M   39M cpu33    1    0   0:00:40 1.2% ctfconvert/4
  5369 af        229M  200M cpu28    1    0   0:00:43 1.3% ctfconvert/4
  5369 af        137M  108M cpu7     1    0   0:00:45 1.4% ctfconvert/4
  5369 af        117M   88M cpu39    1    0   0:00:47 1.3% ctfconvert/1
  5369 af        214M  185M cpu14    1    0   0:00:50 1.4% ctfconvert/4

Following this change, ::findleaks shows none.


Related issues

Related to illumos gate - Bug #13247: CTF conversion fails with large filesClosedAndy Fiddaman

Actions
#1

Updated by Andy Fiddaman about 1 month ago

  • Related to Bug #13247: CTF conversion fails with large files added
#2

Updated by Andy Fiddaman about 1 month ago

  • Description updated (diff)
#3

Updated by Electric Monk about 1 month ago

  • Gerrit CR set to 1015
#4

Updated by Electric Monk 15 days ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit a676209deb2ce5d0c98f331659de25e2483f8c4c

commit  a676209deb2ce5d0c98f331659de25e2483f8c4c
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2020-11-12T21:15:20.000Z

    13252 ctf_update()/ctf_dwarf_convert_function() leak memory
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF