illumos gate

The adt_to_ipv[46]_address functions in libbsm create a Terminal ID for auditing and the at_port member in the resulting data structure is constructed from the client and server ports associated with the UDP/TCP connection; the bottom 32-bits end up being the network-byte-order values of the remote and local port numbers.

If this terminal ID is then used in the argument to setaudit_addr() by a 64-bit binary, subsequent calls to getaudit_addr() by 32-bit apps will fail with EOVERFLOW.

This is because the at_port value is type dev_t which encodes device major and minor numbers differently in 32 & 64-bit. The at_port value constructed in libbsm by a 64-bit app cannot be compressed into 32-bit representation since the minor number is too large. The 32-bit representation uses 14-bits for the major number and 18 for the minor and the minor number that has been constructed needs more than 18 bits (see https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/os/subr.c#L144 for the compression code)

We ran into this under OmniOS after updating OpenSSH to 64-bit. We have a local patch to OpenSSH which uses adt_to_ipv4_address() to construct the audit terminal ID. Logging in via SSH and then running the 32-bit sudo binary prints:

sudo: getaudit_addr: Value too large for defined data type

Also confirmed with the test program below:

bloody% echo $SSH_CLIENT
    x.x.x.x 43711 22
    bloody% echo '43711 16 o p' | dc
    AABF
    bloody% pfexec ./c32 # 32-bit binary
    getaudit_addr: Value too large for defined data type
    bloody% pfexec ./c64 # 64-bit binary
    at_port: ffffffffbfaa1600

source:

#include <stdio.h>
    #include <sys/param.h>
    #include <bsm/libbsm.h>

int
    main()
{
        struct auditinfo_addr info;

if (getaudit_addr(&info, sizeof(info)) != 0)
                perror("getaudit_addr");
        else
    #ifdef _LP64
                printf("at_port: %llx\n", info.ai_termid.at_port);
    #else
                printf("at_port: %lx\n", info.ai_termid.at_port);
    #endif

return 0;
    }

I have a patch for this on which I'll be seeking reviews shortly.