illumos gate

The hex value 4012cccccccccccd is the in-memory representation of the double-precision floating-point value "4.7".[1] And mdb claims to print out double-precision floating-point values:

> ::formats ! grep double
F - double (8 bytes)

It works for 32-bit processes:

$ mdb -p $$
Loading modules: [ ld.so.1 libc.so.1 ]
> ::status
debugging PID 87563 (32-bit)
file: /usr/bin/bash
threading model: native threads
status: stopped by debugger, sleeping in waitid system call
> 4012cccccccccccd::print -i double
+4.7000000e+00
> 4012cccccccccccd=F
                +4.7000000000000002e+00

but it doesn't always work for 64-bit mdb:

$ mdb -p 98267
Loading modules: [ ld.so.1 libumem.so.1 libc.so.1 libproc.so.1 ]
> ::status
debugging PID 98267 (64-bit)
file: /usr/bin/amd64/mdb
threading model: native threads
status: stopped by job control, debugger stop directive pending, sleeping in ioctl
 system call
> 4012cccccccccccd=F
                -NaN

I believe this failure results from the way arguments are passed between mdb_fmt_print(), which implements the format characters, and mdb_iob_printf(), which is used by mdb_fmt_print() to actually print values out. In particular, mdb_iob_printf() uses stdarg-based varargs. On 32-bit, this is fairly straightforward. However, with amd64, this is complicated by the fact that parameters are passed in registers, and which registers are used depends on whether arguments are passed as integer or floating-point values.[2] mdb_fmt_print() treats the value it reads as an opaque integer. In the case of a double, mdb_fmt_print() reads 8 bytes as expected – into a uint64_t. It passes the uint64_t to mdb_iob_printf().

Over in mdb_iob_printf(), the varargs are ultimately processed by iob_doprnt() like this:

                                u.d = VA_ARG(ap, double);

where VA_ARG() will be expanded to the expected va_arg() invocation. Since the amd64 ABI expects doubles to be found in the floating-point registers, but the caller passed the value as an integer, we end up reading garbage and processing that as the value to print, resulting in NaN.

I wrote a tiny dmod to demonstrate the problem:

$ cat mdb_buggy.c 
/*
 * This Source Code Form is subject to the terms of the Mozilla Public
 * License, v. 2.0. If a copy of the MPL was not distributed with this
 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
 */

/*
 * Copyright (c) 2016, Joyent, Inc.
 */

/*
 * mdb_buggy.c: dmod that demonstrates bad uses of varargs in dmods
 */

#include <sys/mdb_modapi.h>
#include <sys/types.h>

static int
dcmd_buggy(uintptr_t addr, uint_t flags, int argc, const mdb_arg_t *argv)
{
        union u {
                double   d;
                uint64_t u64;
        } u;

        u.d = 4.7;
        mdb_printf("%g\n", u.u64);
        mdb_printf("%g\n", u.d);
        return (DCMD_OK);
}

static const mdb_dcmd_t buggy_mdb_dcmds[] = {
        { "buggy", NULL, "buggy", dcmd_buggy },
        { NULL }
};

static mdb_modinfo_t buggy_mdb = { MDB_API_VERSION, buggy_mdb_dcmds, NULL };

const mdb_modinfo_t *
_mdb_init(void)
{
        return (&buggy_mdb);
}

Build and use it like this:

$ gcc -m64 -fPIC -shared -Wall -Wextra -Werror -Wno-unused-parameter -Wno-missing-field-initializers -o mdb_buggy.so mdb_buggy.c
$ mdb
> ::load ./mdb_buggy.so
> ::buggy
-NaN
+4.7000000000000002e+00

In terms of fixing this: mdb_fmt_print() is the varargs call site, and it has no idea that it's even working with a floating-point value because of the way it's been abstracted. Nevertheless, I think it needs to have this knowledge. It might have been simpler to have iob_doprnt() unmarshal the value for a double as though it were a uint64_t (since they're the same size), but I'm not sure that's standards-compliant, and it would totally break users of mdb_printf() that are explicitly using doubles.

[1] You can confirm floating-point hex values at http://babbage.cs.qc.cuny.edu/IEEE-754.old/Decimal.html. I did this using this program:

/*
 * double2bytes.c: print the memory representation of a double-precision
 * floating point number.
 */

#include <err.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>

int
main(int argc, char *argv[])
{
        union {
                double d;
                unsigned long long v64;
        } v;
        char *endp;

        if (argc != 2) {
                errx(2, "usage: double2bytes NUMBER");
        }

        errno = 0;
        v.d = strtod(argv[1], &endp);
        if (v.d == 0 && errno != 0) {
                err(2, "failed to parse argument");
        }
        if (*endp != '\0') {
                errx(2, "failed to parse argument: trailing characters");
        }

        (void) printf("%f = %llx\n", v.d, v.v64);
        return (0);
}

[2] The AMD64 ABI explicitly points out that when using varargs, it's still expected that integer arguments go into the integer registers and floating-point arguments go into the floating-point registers. See http://www.x86-64.org/documentation/abi.pdf. The example is on pages 50-51, figure 3.31 - 3.32.