Bug #3919
closedmdb/pmap should show unresolved map names on core dumps
100%
Description
% LONGPATH=`tail -40 /usr/dict/words | grep -v \\' | nawk '{ printf("%s/", $0)}'` % mkdir -p $LONGPATH % cp /usr/lib/libumem.so.1 $LONGPATH % export LD_LIBRARY_PATH=$LONGPATH % export LD_PRELOAD=libumem.so.1 % cat & % gcore `pgrep -n cat` % pkill -9 -n cat
Take the core file that results from the gcore and note that pmap and mdb both behave correctly:
% pmap core.40191 ... FEF10000 196K r-x-- /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zounds/zucchini/Zurich/zygote/libumem.so.1 FEF51000 76K rw--- /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zounds/zucchini/Zurich/zygote/libumem.so.1 FEF64000 24K rw--- /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zounds/zucchini/Zurich/zygote/libumem.so.1 ... % mdb core.40191 > $m ... fef10000 fef41000 31000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zounds/zucchini/Zurich/zygote/libumem.so.1 fef51000 fef64000 13000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zounds/zucchini/Zurich/zygote/libumem.so.1 fef64000 fef6a000 6000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zounds/zucchini/Zurich/zygote/libumem.so.1 ...
Now delete the libumem.so.1:
% rm $LONGPATH/libumem.so.1
Now pmap won't find any libumem.so.1 at all:
# pmap core.40191 core 'core.40191' of 40191: cat 08046000 8K rw--- [ stack ] 08050000 68K r-x-- /opt/local/bin/gcat 08070000 8K rwx-- /opt/local/bin/gcat 08072000 320K rwx-- [ heap ] FEA30000 64K rwx-- [ anon ] FEB3E000 4K rw--- [ anon ] FEB40000 4K r-x-- /lib/libumem_trampoline.so.1 FEB51000 4K rw--- /lib/libumem_trampoline.so.1 FEB60000 12K rwx-- /lib/libumem_trampoline.so.1 FEBD0000 64K rw--- [ anon ] FEBE7000 128K rw--- [ anon ] FEC08000 4K rwx-- [ anon ] FEC0C000 76K r---- [ anon ] FEC20000 4K rwx-- [ anon ] FEC30000 24K rwx-- [ anon ] FEC40000 100K r-x-- /opt/local/gcc47/i386-sun-solaris2.11/lib/libgcc_s.so.1 FEC68000 4K rwx-- /opt/local/gcc47/i386-sun-solaris2.11/lib/libgcc_s.so.1 FEC70000 4K rwx-- [ anon ] FEC80000 1232K r-x-- /lib/libc.so.1 FEDC4000 36K rwx-- /lib/libc.so.1 FEDCD000 8K rwx-- /lib/libc.so.1 FEDD0000 904K r-x-- /opt/local/lib/libiconv.so.2.5.1 FEEC1000 8K rwx-- /opt/local/lib/libiconv.so.2.5.1 FEED0000 4K rwx-- [ anon ] FEEE0000 36K r-x-- /opt/local/lib/libintl.so.8.1.1 FEEF8000 4K rwx-- /opt/local/lib/libintl.so.8.1.1 FEF00000 4K rwx-- [ anon ] FEF10000 196K r-x-- [ anon ] FEF51000 76K rw--- [ anon ] FEF64000 24K rw--- [ anon ] FEF70000 4K r----* [ anon ] FEF80000 4K rw--- [ anon ] FEF90000 4K rw--- [ anon ] FEFA0000 4K rwx-- [ anon ] FEFB0000 4K rwx-- [ anon ] FEFB6000 212K r-x-- /lib/ld.so.1 FEFFB000 8K rwx-- /lib/ld.so.1 FEFFD000 4K rwx-- /lib/ld.so.1 total 3676K
mdb has the path, but it's been truncated to PRMAPSZ (64 bytes):
> $m BASE LIMIT SIZE NAME 2000 [ stack ] 11000 /opt/local/bin/gcat 2000 /opt/local/bin/gcat 50000 [ heap ] fea30000 fea40000 10000 feb3e000 feb3f000 1000 feb40000 feb41000 1000 LMfef00428`/lib/libumem_trampoline.so.1 feb51000 feb52000 1000 LMfef00428`/lib/libumem_trampoline.so.1 feb60000 feb63000 3000 LMfef00428`/lib/libumem_trampoline.so.1 febd0000 febe0000 10000 febe7000 fec07000 20000 fec08000 fec09000 1000 fec0c000 fec1f000 13000 fec20000 fec21000 1000 fec30000 fec36000 6000 fec40000 fec59000 19000 /opt/local/gcc47/i386-sun-solaris2.11/lib/libgcc_s.so.1 fec68000 fec69000 1000 /opt/local/gcc47/i386-sun-solaris2.11/lib/libgcc_s.so.1 fec70000 fec71000 1000 fec80000 fedb4000 134000 /lib/libc.so.1 fedc4000 fedcd000 9000 /lib/libc.so.1 fedcd000 fedcf000 2000 /lib/libc.so.1 fedd0000 feeb2000 e2000 /opt/local/lib/libiconv.so.2.5.1 feec1000 feec3000 2000 /opt/local/lib/libiconv.so.2.5.1 feed0000 feed1000 1000 feee0000 feee9000 9000 /opt/local/lib/libintl.so.8.1.1 feef8000 feef9000 1000 /opt/local/lib/libintl.so.8.1.1 fef00000 fef01000 1000 fef10000 fef41000 31000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/ziggi fef51000 fef64000 13000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/ziggi fef64000 fef6a000 6000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/ziggi fef70000 fef71000 1000 fef80000 fef81000 1000 fef90000 fef91000 1000 fefa0000 fefa1000 1000 fefb0000 fefb1000 1000 fefb6000 fefeb000 35000 /lib/ld.so.1 feffb000 feffd000 2000 /lib/ld.so.1 feffd000 feffe000 1000 /lib/ld.so.1
But mdb won't find any libumem.so.1 symbols:
> ::nm ! grep umem_alloc mdb: unable to dump symbol data for: /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/ziggi
There are multiple ways to fix this issue, but by far the simplest (and most broadly useful) is to fix this issue by fixing a broader issue: that mdb (and pmap) try to resolve linker names on a core dump. Both of these commands have options for the unresolved behavior ("-l" for pmap and the extraordinarily obscure "lmraw" option for mdb); these options should be enabled by default when debugging postmortem. When this is done, the $m for the above dump has the true name length:
BASE LIMIT SIZE NAME 8046000 8048000 2000 [ stack ] 8050000 8061000 11000 /opt/local/bin/gcat 8070000 8072000 2000 /opt/local/bin/gcat 8072000 80c2000 50000 [ heap ] fea30000 fea40000 10000 feb3e000 feb3f000 1000 feb40000 feb41000 1000 LMfef00428`/lib/libumem_trampoline.so.1 feb51000 feb52000 1000 LMfef00428`/lib/libumem_trampoline.so.1 feb60000 feb63000 3000 LMfef00428`/lib/libumem_trampoline.so.1 febd0000 febe0000 10000 febe7000 fec07000 20000 fec08000 fec09000 1000 fec0c000 fec1f000 13000 fec20000 fec21000 1000 fec30000 fec36000 6000 fec40000 fec59000 19000 /opt/local/gcc47/i386-sun-solaris2.11/lib/libgcc_s.so.1 fec68000 fec69000 1000 /opt/local/gcc47/i386-sun-solaris2.11/lib/libgcc_s.so.1 fec70000 fec71000 1000 fec80000 fedb4000 134000 /lib/libc.so.1 fedc4000 fedcd000 9000 /lib/libc.so.1 fedcd000 fedcf000 2000 /lib/libc.so.1 fedd0000 feeb2000 e2000 /opt/local/lib/libiconv.so.2.5.1 feec1000 feec3000 2000 /opt/local/lib/libiconv.so.2.5.1 feed0000 feed1000 1000 feee0000 feee9000 9000 /opt/local/lib/libintl.so.8.1.1 feef8000 feef9000 1000 /opt/local/lib/libintl.so.8.1.1 fef00000 fef01000 1000 fef10000 fef41000 31000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zounds/zucchini/Zurich/zygote/libumem.so.1 fef51000 fef64000 13000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zo unds/zucchini/Zurich/zygote/libumem.so.1 fef64000 fef6a000 6000 /root/zero/zeroes/zeroth/zest/zesty/zeta/Zeus/Ziegler/zig/zigging/zigzag/zigzagging/zilch/Zimmerman/zinc/zing/Zion/Zionism/zip/zippy/zircon/zirconium/zloty/zodiac/zodiacal/Zoe/Zomba/zombie/zone/zoo/zoology/zoom/Zorn/Zoroaster/Zoroastrian/zo unds/zucchini/Zurich/zygote/libumem.so.1 fef70000 fef71000 1000 fef80000 fef81000 1000 fef90000 fef91000 1000 fefa0000 fefa1000 1000 fefb0000 fefb1000 1000 fefb6000 fefeb000 35000 /lib/ld.so.1 feffb000 feffd000 2000 /lib/ld.so.1 feffd000 feffe000 1000 /lib/ld.so.1
Updated by Robert Mustacchi almost 9 years ago
- Status changed from New to Resolved
- % Done changed from 90 to 100
Resolved in 8f68126c6dc4f2178929233f0c7ccce877e973ca.