illumos gate

Some selections from the Delphix bug tracker:

Madhav Suresh:

after a kernel panic, running ::dtrace after using dtrace print() results in
gobbledygook, or no ouput

Dan Kimmel:

When I save a core file during a dtrace session that is using print(), and then
use ::dtrace to analyze the core file, mdb fails with the following SIGSEGV:

> ::dtrace_state
            ADDR MINOR             PROC NAME                         FILE
ffffff01ca5547c0     2 ffffff01cc029030 dtrace           ffffff01ca96dbf0
ffffff01e01ecc80     3 ffffff01cc1f0070 dtrace           ffffff01ca50b6a0
> ffffff01e01ecc80::dtrace
CPU     ID                    FUNCTION:NAME

*** mdb: received signal SEGV at:
    [1] libc.so.1`_fprintf+0x92()
    [2] libdtrace.so.1`dt_print_indent+0x29()
    [3] libdtrace.so.1`dt_print_member+0x11f()
    [4] libctf.so.1`ctf_type_rvisit+0x69()
    [5] libctf.so.1`ctf_type_visit+0x19()
    [6] libdtrace.so.1`dtrace_print+0xf5()
    [7] libdtrace.so.1`dt_consume_cpu+0x4b5()
    [8] libdtrace.so.1`dtrace_consume+0x3f7()
    [9] dtrace.so`dtrace+0x280()
    [10] mdb`dcmd_invoke+0x64()
    [11] mdb`mdb_call_idcmd+0xff()
    [12] mdb`mdb_call+0x410()
    [13] mdb`yyparse+0x50d()
    [14] mdb`mdb_run+0x2cd()
    [15] mdb`main+0x136d()
    [16] mdb`_start+0x6c()

Disassembling the function shows us this line:
libc.so.1`fprintf+0x92:         movl   0x24(%r13),%eax

When I save a core file during a dtrace session that is using print(), and then
use ::dtrace to analyze the core file, mdb fails with the following SIGSEGV:

> ::dtrace_state
            ADDR MINOR             PROC NAME                         FILE
ffffff01ca5547c0     2 ffffff01cc029030 dtrace           ffffff01ca96dbf0
ffffff01e01ecc80     3 ffffff01cc1f0070 dtrace           ffffff01ca50b6a0
> ffffff01e01ecc80::dtrace
CPU     ID                    FUNCTION:NAME

*** mdb: received signal SEGV at:
    [1] libc.so.1`_fprintf+0x92()
    [2] libdtrace.so.1`dt_print_indent+0x29()
    [3] libdtrace.so.1`dt_print_member+0x11f()
    [4] libctf.so.1`ctf_type_rvisit+0x69()
    [5] libctf.so.1`ctf_type_visit+0x19()
    [6] libdtrace.so.1`dtrace_print+0xf5()
    [7] libdtrace.so.1`dt_consume_cpu+0x4b5()
    [8] libdtrace.so.1`dtrace_consume+0x3f7()
    [9] dtrace.so`dtrace+0x280()
    [10] mdb`dcmd_invoke+0x64()
    [11] mdb`mdb_call_idcmd+0xff()
    [12] mdb`mdb_call+0x410()
    [13] mdb`yyparse+0x50d()
    [14] mdb`mdb_run+0x2cd()
    [15] mdb`main+0x136d()
    [16] mdb`_start+0x6c()

Disassembling the function shows us this line:
libc.so.1`fprintf+0x92:         movl   0x24(%r13),%eax

This was caused because dtrace() passes a NULL FILE* to dtrace_consume(). I'm
trying to figure out why it does that.

And Dan's final analysis before fixing the bug:

Many modules in mdb (including the one in dtrace.c) are separated from the
internal implementation of mdb, so they can call public methods like
mdb_printf(), but they can't access the "mdb" variable which contains the FILE*
necessary to call fprintf(mdb.m_out, ...) directly. I'm not exactly sure why
this is (other than general encapsulation), but there's probably a good reason
for it because the compiler /really/ doesn't like it when I try to include
<mdb/mdb.h> with _MDB_PRIVATE defined in dtrace.c.

We can't simply pass in stdout to dtrace_consume(), however, since
dtrace_print() would end up sending its output to the wrong place if mdb.m_out
!= stdout. We also can't make dtrace_print() use mdb_printf() instead of
fprintf() to implement print() because that would create a dependency from
libdtrace to mdb. Finally, we can't modify the signature of dtrace() to pass in
mdb.m_out because it's a callback (with many others of the same type).

I'm going to take one of two paths (preferably 1):
1. Find/implement dtrace_print() using snprintf() or similar instead of
fprintf(). Then, write the filled-in buffer using mdb_printf() in dtrace.c.
2. Use a temporary FILE* to collect output from the dtrace_consume() call in
dtrace(), and copy the output into the correct buffer using mdb_printf() in
dtrace.c, releasing the FILE* afterward. This is a bit hackier but is
guaranteed to work.