Project

General

Profile

Actions

Feature #13925

closed

core files should include DWARF

Added by Joshua M. Clulow 5 months ago. Updated 19 days ago.

Status:
Closed
Priority:
Normal
Category:
kernel
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Expert
Tags:
Gerrit CR:

Description

At present, when a user process dumps core the kernel copies the symbol table and any CTF information from the object files of the executing process into the core file. In this way, debuggers like MDB have access to the correct debugging information for a core file, without needed to retain and correlate with the executable or libraries, or any ancillary debug information files.

If an executable, or any dependent library, includes DWARF or any other debugging information, we should copy it to the core file as we do with the symbols and CTF.

For example, these sections are included in a Rust program built with debug information:

$ elfdump -c target/debug/sample-server |grep '\.debug'
Section Header[19]:  sh_name: .debug_gdb_scripts
Section Header[7800]:  sh_name: .debug_abbrev
Section Header[7801]:  sh_name: .debug_info
Section Header[7802]:  sh_name: .debug_aranges
Section Header[7803]:  sh_name: .debug_str
Section Header[7804]:  sh_name: .debug_pubnames
Section Header[7805]:  sh_name: .debug_pubtypes
Section Header[7807]:  sh_name: .debug_line
Section Header[7808]:  sh_name: .debug_ranges
Section Header[7809]:  sh_name: .debug_loc
Section Header[7810]:  sh_name: .debug_frame
Actions #1

Updated by Robert Mustacchi 4 months ago

  • Subject changed from core files should include DWARF from the executable to core files should include DWARF
Actions #2

Updated by Electric Monk 2 months ago

  • Gerrit CR set to 1662
Actions #3

Updated by Robert Mustacchi 25 days ago

I've tested this in a bunch of different ways. First, let me talk about the automated testing. I wrote a general test suite for testing coreadm settings and tested different variants of having things that we construct in the shstrtab that is generated in the core file. In particular, this makes sure to exercise that we have sections for different things including:

  • Symbol Tables
  • CTF sections
  • Dwarf sections
  • And other variations thereof that include things like the stacks, heaps, etc.

For each of these we make sure that we get a core with the expected core content from both the kernel and via gcore. In addition, it goes through and makes sure of the following:

  • The libraries in the test that have dwarf, ctf, symbol tables, etc. are all present based on the expected content
  • When present and there is enough information to map these back to a source library, the elf data in the corresponding sections is identical (that is the symbol table, ctf, and dwarf sections actually match between the core and related). This also ensures that we have the VA mappings on sections present in the core files that match up. This is a way of making sure that we actually have multiple instances of the dwarf data (similar to CTF and symbol tables).

The following is the test suite run:

$ pfexec /opt/os-tests/tests/cores/coretests 
TEST PASSED: kernel dumper.32 none
TEST PASSED: gcore dumper.32 none
TEST PASSED: kernel dumper.64 none
TEST PASSED: gcore dumper.64 none
TEST PASSED: kernel dumper.32 ctf
TEST PASSED: gcore dumper.32 ctf
TEST PASSED: kernel dumper.64 ctf
TEST PASSED: gcore dumper.64 ctf
TEST PASSED: kernel dumper.32 debug
TEST PASSED: gcore dumper.32 debug
TEST PASSED: kernel dumper.64 debug
TEST PASSED: gcore dumper.64 debug
TEST PASSED: kernel dumper.32 symtab
TEST PASSED: gcore dumper.32 symtab
TEST PASSED: kernel dumper.64 symtab
TEST PASSED: gcore dumper.64 symtab
TEST PASSED: kernel dumper.32 ctf+debug+symtab
TEST PASSED: gcore dumper.32 ctf+debug+symtab
TEST PASSED: kernel dumper.64 ctf+debug+symtab
TEST PASSED: gcore dumper.64 ctf+debug+symtab
TEST PASSED: kernel dumper.32 anon+data+ctf+debug+symtab
TEST PASSED: gcore dumper.32 anon+data+ctf+debug+symtab
TEST PASSED: kernel dumper.64 anon+data+ctf+debug+symtab
TEST PASSED: gcore dumper.64 anon+data+ctf+debug+symtab
TEST PASSED: kernel dumper.32 default
TEST PASSED: gcore dumper.32 default
TEST PASSED: kernel dumper.64 default
TEST PASSED: gcore dumper.64 default
TEST PASSED: kernel dumper.32 default-ctf-debug-symtab
TEST PASSED: gcore dumper.32 default-ctf-debug-symtab
TEST PASSED: kernel dumper.64 default-ctf-debug-symtab
TEST PASSED: gcore dumper.64 default-ctf-debug-symtab
TEST PASSED: kernel dumper.32 default+debug
TEST PASSED: gcore dumper.32 default+debug
TEST PASSED: kernel dumper.64 default+debug
TEST PASSED: gcore dumper.64 default+debug
TEST PASSED: kernel dumper.32 default-symtab
TEST PASSED: gcore dumper.32 default-symtab
TEST PASSED: kernel dumper.64 default-symtab
TEST PASSED: gcore dumper.64 default-symtab

I also ran that as part of the broader test suite:

rm@beowulf:~$ pfexec /opt/os-tests/bin/ostest 
Test: /opt/os-tests/tests/clock_gettime.32 (run as root)          [00:00] [PASS]
Test: /opt/os-tests/tests/clock_gettime.64 (run as root)          [00:00] [PASS]
Test: /opt/os-tests/tests/eventfd.32 (run as root)                [00:00] [PASS]
Test: /opt/os-tests/tests/eventfd.64 (run as root)                [00:00] [PASS]
Test: /opt/os-tests/tests/imc_test (run as root)                  [00:00] [PASS]
Test: /opt/os-tests/tests/odirectory.32 (run as root)             [00:00] [PASS]
Test: /opt/os-tests/tests/odirectory.64 (run as root)             [00:00] [PASS]
Test: /opt/os-tests/tests/writev.32 (run as root)                 [00:00] [PASS]
Test: /opt/os-tests/tests/writev.64 (run as root)                 [00:00] [PASS]
Test: /opt/os-tests/tests/cores/core_prereqs (run as root)        [00:00] [PASS]
Test: /opt/os-tests/tests/cores/coretests (run as root)           [00:04] [PASS]
Test: /opt/os-tests/tests/ddi_ufm/ufm-test-setup (run as root)    [00:00] [PASS]
Test: /opt/os-tests/tests/ddi_ufm/ufm-test (run as root)          [00:00] [PASS]
Test: /opt/os-tests/tests/ddi_ufm/ufm-test-cleanup (run as root)  [00:00] [PASS]
Test: /opt/os-tests/tests/file-locking/runtests.32 (run as root)  [00:21] [PASS]
Test: /opt/os-tests/tests/file-locking/runtests.64 (run as root)  [00:21] [PASS]
Test: /opt/os-tests/tests/i386/ldt (run as root)                  [00:06] [PASS]
Test: /opt/os-tests/tests/i386/badseg (run as root)               [00:06] [PASS]
Test: /opt/os-tests/tests/ksensor/ksensor_init (run as root)      [00:00] [PASS]
Test: /opt/os-tests/tests/ksensor/ksensor_basic.32 (run as root)  [00:00] [PASS]
Test: /opt/os-tests/tests/ksensor/ksensor_basic.64 (run as root)  [00:00] [PASS]
Test: /opt/os-tests/tests/ksensor/ksensor_err.32 (run as root)    [00:00] [PASS]
Test: /opt/os-tests/tests/ksensor/ksensor_err.64 (run as root)    [00:00] [PASS]
Test: /opt/os-tests/tests/ksensor/ksensor_fini (run as root)      [00:00] [PASS]
Test: /opt/os-tests/tests/libtopo/digraph-test (run as root)      [00:00] [PASS]
Test: /opt/os-tests/tests/pf_key/acquire-compare (run as root)    [00:43] [PASS]
Test: /opt/os-tests/tests/pf_key/kmc-update (run as root)         [00:00] [PASS]
Test: /opt/os-tests/tests/sdevfs/sdevfs_eisdir (run as root)      [00:00] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_aslr (run as root)    [00:02] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_core (run as root)    [00:01] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_dts (run as root)     [00:02] [FAIL]
Test: /opt/os-tests/tests/secflags/secflags_elfdump (run as root) [00:01] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_forbidnullmap (run as root) [00:00] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_limits (run as root)  [00:00] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_noexecstack (run as root) [00:00] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_proc (run as root)    [00:01] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_psecflags (run as root) [00:11] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_syscall (run as root) [00:00] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_truss (run as root)   [00:00] [PASS]
Test: /opt/os-tests/tests/secflags/secflags_zonecfg (run as root) [00:00] [PASS]
Test: /opt/os-tests/tests/sigqueue/sigqueue_queue_size (run as root) [00:00] [PASS]
Test: /opt/os-tests/tests/sockfs/conn (run as root)               [00:05] [PASS]
Test: /opt/os-tests/tests/sockfs/dgram (run as root)              [00:05] [PASS]
Test: /opt/os-tests/tests/sockfs/drop_priv (run as root)          [00:07] [PASS]
Test: /opt/os-tests/tests/sockfs/nosignal (run as root)           [00:02] [PASS]
Test: /opt/os-tests/tests/sockfs/rights.32 (run as root)          [00:00] [PASS]
Test: /opt/os-tests/tests/sockfs/rights.64 (run as root)          [00:00] [PASS]
Test: /opt/os-tests/tests/sockfs/sockpair (run as root)           [00:10] [PASS]
Test: /opt/os-tests/tests/sockfs/recvmsg.32 (run as root)         [00:00] [PASS]
Test: /opt/os-tests/tests/sockfs/recvmsg.64 (run as root)         [00:00] [PASS]
Test: /opt/os-tests/tests/stackalign/stackalign.32 (run as root)  [00:00] [PASS]
Test: /opt/os-tests/tests/stackalign/stackalign.64 (run as root)  [00:00] [PASS]
Test: /opt/os-tests/tests/stress/dladm-kstat (run as root)        [00:20] [PASS]
Test: /opt/os-tests/tests/syscall/open.32 (run as root)           [00:00] [PASS]
Test: /opt/os-tests/tests/syscall/open.64 (run as root)           [00:00] [PASS]
Test: /opt/os-tests/tests/timer/timer_limit (run as root)         [00:00] [PASS]
Test: /opt/os-tests/tests/uccid/atrparse (run as root)            [00:00] [PASS]

Results Summary
FAIL       1
PASS      56

Running Time:    00:02:53
Percent passed:    98.2%
Log directory:    /var/tmp/test_results/20211111T130251

Finally, for manual testing, I've gone through and taken the cores and used various tools to print some of the DWARF sections. I also checked that findleaks of a gcore was empty and findleaks on a kernel dump had nothing here. The secflags_dts error is due to #14229. With WIP fixes for that, I find that it passes.

Actions #4

Updated by Electric Monk 19 days ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 8e458de0baeb1fee50643403223bc7e909a48464

commit  8e458de0baeb1fee50643403223bc7e909a48464
Author: Robert Mustacchi <rm@fingolfin.org>
Date:   2021-11-11T22:12:38.000Z

    13925 core files should include DWARF
    Reviewed by: Rich Lowe <richlowe@richlowe.net>
    Reviewed by: C Fraire <cfraire@me.com>
    Reviewed by: Adam Leventhal <adam.leventhal@gmail.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Actions

Also available in: Atom PDF