illumos gate

After a little bit more thinking about this I decided to do not allow the authloopback_marshal() to succeed in a case the user is in a large number of groups by encoding only limited set of groups (similarly as authkern_marshal() does today). If I would do that it might confuse the AUTH_LOOPBACK users when they would receive incomplete list of groups. I believe the current AUTH_LOOPBACK consumers in the gate are not prone to be confused by that, but still it would be better to do not change the current behavior to be safe.

In any case, if needed, we can easily change the authloopback_marshal() behavior in future to behave similarly as authkern_marshal() does and do not fail for large number of groups.

To reproduce this problem compile the attached module.c file using these steps:

$ /opt/gcc/4.4.4/bin/gcc -Wall -D_KERNEL -m64 -mcmodel=kernel -mno-red-zone -ffreestanding -nodefaultlibs -c module.c
$ /usr/ccs/bin/ld -dy -N rpcsec -r -o module module.o

and then do this:

# mdb -kwe 'ngroups_max/W 100'
# hostname $(python -c 'print "x" * 100')
# modload module

You will see this in the log:

Apr 26 16:00:37 t4 module: [ID 378104 kern.info] NOTICE: AUTH_LOOPBACK, 70 groups, xdrmem success: 000055de 00000190
Apr 26 16:00:37 t4 module: [ID 618828 kern.info] NOTICE: AUTH_LOOPBACK, 70 groups, xdrmblk success: 000055de 000000a4
Apr 26 16:00:37 t4 module: [ID 633239 kern.info] NOTICE: AUTH_LOOPBACK, 80 groups, xdrmem success: 000055de 000001b8
Apr 26 16:00:37 t4 module: [ID 767423 kern.info] NOTICE: AUTH_LOOPBACK, 80 groups, xdrmblk success: 000055de 000000a4
Apr 26 16:00:37 t4 module: [ID 398392 kern.info] NOTICE: AUTH_SYS, 80 groups, xdrmem success: 00000001 000000b8
Apr 26 16:00:37 t4 module: [ID 301671 kern.info] NOTICE: AUTH_SYS, 80 groups, xdrmblk success: 00000001 000000a4

At line 3 you see the encoded size of the auth body 0x1b8 (440) bytes. For this case (and also for line 4) it was expected that the AUTH_MARSHALL() call would fail.

git commit 6dd72a43d2e43185833c20e7f0c4cb88a4d37ec8

commit  6dd72a43d2e43185833c20e7f0c4cb88a4d37ec8
Author: Marcel Telka <marcel@telka.sk>
Date:   2017-06-13T14:37:46.000Z

    8106 authloopback_marshal() can violate the RPC specification
    8109 Kernel AUTH_SYS and AUTH_LOOPBACK implementation can ignore provided credentials
    Reviewed by: Toomas Soome <tsoome@me.com>
    Reviewed by: Jason King <jason.brian.king+illumos@gmail.com>
    Approved by: Dan McDonald <danmcd@joyent.com>