getupeercred causes spurious event port wakeups on FIFOs
It turns out that event ports are slightly broken for FIFOs.
Specifically, it is possible for
port_getn to return an event on a FIFO fd (registered via
POLLIN) even though there is nothing available for reading.
Attached is a minimal test case.
If you run it with stdin being a file or a tty,
port_getn blocks until bytes are available for reading:
$ ./a.out port = 3, errno = 0 ret = 0, errno = 0 ret = -1, errno = 48 ucred = 803bc88 <waiting>
If, however, stdin is a pipe,
port_getn returns immediately because the fd is marked as readable by the
$ cat | ./a.out port = 3, errno = 0 ret = 0, errno = 0 ret = -1, errno = 48 ucred = 803e258 ret = 0, errno = 48 ev.portev_source = 4 ev.portev_events = 1 ev.portev_object = 0 ev.portev_user = 0
pfiles(1) injects a
getpeerucred call into the target with the help of the agent lwp.
getpeerucred call ends up issuing an ioctl on the FIFO. Since the
_I_GETPEERCRED command is not handled by fastpath mode (http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/fifofs/fifovnops.c#1160), the FIFO is converted to a streams mode FIFO. During this conversion, all poll and event ports waiters are awoken to cause them to re-register with the newly converted FIFO (http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/fifofs/fifosubr.c#1108). The kernel stack of a thread waking up all waiters because of the FIFO conversion looks like this:
genunix`port_send_event+0x131 genunix`pollwakeup+0x86 genunix`strpollwakeup+0x20 fifofs`fifo_fastturnoff+0xae fifofs`fifo_fastoff+0x92 fifofs`fifo_fastioctl+0x19d fifofs`fifo_ioctl+0x3d genunix`fop_ioctl+0x55 genunix`getpeerucred+0xff genunix`ucredsys+0x52 genunix`ucredsys32+0x1b unix`sys_syscall32+0xff
This spurious wakeup can cause userspace applications to end up blocking on a subsequent read/write because they were notified that the file descriptor was ready. (This problem has been spotted in the wild: https://github.com/PowerDNS/pdns/issues/2925) The manpages for event ports don't specify whether spurious events are something an application needs to guard against.
As far as I can tell, there appears to be no good reason for fastpath FIFOs to not handle
_I_GETPEERCRED ioctls directly. Attached is a quick patch which appears to work with my quick test.