illumos gate

From Bryan's bug report at Joyent:

Building 32-bit ustack helpers on bits that include the fix to illumos#3024 results in helpers that have (silent) errors. As it turns out, this problem can be demonstrated much more simply (that is, without involving helpers); the following works:

dtrace -n BEGIN'{*((int *)alloca(4)) = 1; exit(0)}'

But this does not:

# dtrace -32 -qn BEGIN'{*((int *)alloca(4)) = 1; exit(0)}'
dtrace: error on enabled probe ID 1 (ID 1: dtrace:::BEGIN): invalid address (0xb0574010) in action #1 at DIF offset 32

Looking at the DIF shows that narrowing is taking place:

# dtrace -32 -qn BEGIN'{*((int *)alloca(4)) = 1; exit(0)}' -Se

DIFO 0x60c0d0 returns D type (integer) (size 4)
OFF OPCODE      INSTRUCTION
00: 25000001    setx DT_INTEGER[0], %r1         ! 0x1
01: 25000102    setx DT_INTEGER[1], %r2         ! 0x4
02: 33000000    flushts
03: 31000002    pushtv DT_TYPE(0), %r2          ! DT_TYPE(0) = D type
04: 2f000f02    call DIF_SUBR(15), %r2          ! alloca
05: 25000203    setx DT_INTEGER[2], %r3         ! 0x20
06: 04020302    sll  %r2, %r3, %r2
07: 05020302    srl  %r2, %r3, %r2
08: 3e010002    stw  %r1, [%r2]
09: 23000001    ret  %r1

In this case, no narrowing should be occurring -- even though they are 64-bit quantities, the compiler should believe both the return value of alloca() and the result of the type coercion to be unsigned 32-bit quantities. The root cause is an inverted test in dt_cg_typecast(); diffs for a fix:

--- a/usr/src/lib/libdtrace/common/dt_cg.c
+++ b/usr/src/lib/libdtrace/common/dt_cg.c
@@ -476,7 +476,7 @@ dt_cg_typecast(const dt_node_t *src, const dt_node_t *dst,
        if (!dt_node_is_scalar(dst))
                return; /* not a scalar */
        if (dstsize == srcsize &&
-           ((src->dn_flags ^ dst->dn_flags) & DT_NF_SIGNED) != 0)
+           ((src->dn_flags ^ dst->dn_flags) & DT_NF_SIGNED) == 0)
                return; /* not narrowing or changing signed-ness */
        if (dstsize > srcsize && (src->dn_flags & DT_NF_SIGNED) == 0)
                return; /* nothing to do in this case */