illumos gate

Looks like Samba 4.9.0 also wants it now, I get link errors on this. I opened https://bugzilla.samba.org/show_bug.cgi?id=13629 for this.

To test the new memory streams I did a combination of a couple of different things:

• I imported several tests from OpenBSD to cover some basic behavior of open_memstream(3C) and fmemopen(3C). I also used several tests that they had put together for other parts of stdio.
• I wrote out my own unit tests for open_memsrteam(3C), open_wmemstream(3C), and fmemopen(3C) that covered more of the corner case behaviors and other conditions like the failure to allocate memory.
• I made sure that all of the tests employ umem debugging to catch for memory corruption issues.
• I updated the symbols tests to cover the new functions.
• I exercised the resulting system using a couple of different languages and tools to make sure that stdio wasn't totally broken.
• Compared some of the behaviors that we have to glibc, musl, and the BSDs
• I also run the following other test suites (mostly in case they exercised libc/stdio):
  • elf-tests
  • parts of crypto-tests (some tests time out because my laptop CPU isn't fast enough for the current time out)
  • os-tests
  • util-tests (spurious failures from lacking ctf tools, etc.)

Here's a run of the full libc test suite:

rm@nienor:/opt/libc-tests$ pfexec ./bin/libctest 
Test: /opt/libc-tests/tests/aligned_alloc.32 (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/aligned_alloc.64 (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/c11_threads.32 (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/c11_threads.64 (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/c11_tss.32 (run as root)              [00:00] [PASS]
Test: /opt/libc-tests/tests/c11_tss.64 (run as root)              [00:00] [PASS]
Test: /opt/libc-tests/tests/call_once.32 (run as root)            [00:00] [PASS]
Test: /opt/libc-tests/tests/call_once.64 (run as root)            [00:00] [PASS]
Test: /opt/libc-tests/tests/catopen (run as root)                 [00:00] [PASS]
Test: /opt/libc-tests/tests/endian.32 (run as root)               [00:00] [PASS]
Test: /opt/libc-tests/tests/endian.64 (run as root)               [00:00] [PASS]
Test: /opt/libc-tests/tests/env-7076.32 (run as root)             [00:00] [PASS]
Test: /opt/libc-tests/tests/env-7076.64 (run as root)             [00:00] [PASS]
Test: /opt/libc-tests/tests/fnmatch.32 (run as root)              [00:00] [PASS]
Test: /opt/libc-tests/tests/fnmatch.64 (run as root)              [00:00] [PASS]
Test: /opt/libc-tests/tests/fpround_test (run as root)            [00:00] [PASS]
Test: /opt/libc-tests/tests/memset_s.32 (run as root)             [00:00] [PASS]
Test: /opt/libc-tests/tests/memset_s.64 (run as root)             [00:00] [PASS]
Test: /opt/libc-tests/tests/newlocale_test (run as root)          [00:01] [PASS]
Test: /opt/libc-tests/tests/nl_langinfo_test (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/posix_memalign.32 (run as root)       [00:00] [PASS]
Test: /opt/libc-tests/tests/posix_memalign.64 (run as root)       [00:00] [PASS]
Test: /opt/libc-tests/tests/printf-6961.64 (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/printf-9511.32 (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/printf-9511.64 (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/priv_gettext (run as root)            [00:00] [PASS]
Test: /opt/libc-tests/tests/psignal (run as root)                 [00:00] [PASS]
Test: /opt/libc-tests/tests/pthread_attr_get_np (run as root)     [00:00] [PASS]
Test: /opt/libc-tests/tests/quick_exit (run as root)              [00:00] [PASS]
Test: /opt/libc-tests/tests/random/arc4key.ksh (run as root)      [00:03] [PASS]
Test: /opt/libc-tests/tests/random/arc4random (run as root)       [00:00] [PASS]
Test: /opt/libc-tests/tests/random/arc4random_fork (run as root)  [00:00] [PASS]
Test: /opt/libc-tests/tests/random/arc4random_forkall (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/random/arc4random_forksig (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/random/arc4random_prefork (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/random/arc4random_preforkall (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/random/arc4random_preforksig (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/random/chacha (run as root)           [00:00] [PASS]
Test: /opt/libc-tests/tests/random/getentropy (run as root)       [00:00] [PASS]
Test: /opt/libc-tests/tests/random/getrandom (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/random/inz_child (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/random/inz_inval (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/random/inz_mlock (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/random/inz_region (run as root)       [00:00] [PASS]
Test: /opt/libc-tests/tests/random/inz_split (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/random/inz_split_vpp (run as root)    [00:00] [PASS]
Test: /opt/libc-tests/tests/random/inz_vpp (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/regex/regex_test (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/select/select.sh (run as root)        [00:08] [PASS]
Test: /opt/libc-tests/tests/set_constraint_handler_s.32 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/set_constraint_handler_s.64 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/fileno.32 (run as root)         [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/fileno.64 (run as root)         [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/fmemopentest.32 (run as root)   [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/fmemopentest.64 (run as root)   [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/ftell_ungetc.32 (run as root)   [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/ftell_ungetc.64 (run as root)   [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/memstream.32 (run as root)      [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/memstream.64 (run as root)      [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/memstream_reopen.32 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/memstream_reopen.64 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/open_memstreamtest.32 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/open_memstreamtest.64 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/orientation_test.32 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/orientation_test.64 (run as root) [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/test_mbrtowc.32 (run as root)   [00:00] [PASS]
Test: /opt/libc-tests/tests/stdio/test_mbrtowc.64 (run as root)   [00:00] [PASS]
Test: /opt/libc-tests/tests/strcoll-strxfrm-6907.32 (run as root) [00:01] [PASS]
Test: /opt/libc-tests/tests/strcoll-strxfrm-6907.64 (run as root) [00:01] [PASS]
Test: /opt/libc-tests/tests/strerror (run as root)                [00:00] [PASS]
Test: /opt/libc-tests/tests/thread_name (run as root)             [00:00] [PASS]
Test: /opt/libc-tests/tests/timespec_get.32 (run as root)         [00:00] [PASS]
Test: /opt/libc-tests/tests/timespec_get.64 (run as root)         [00:00] [PASS]
Test: /opt/libc-tests/tests/wcsncasecmp-7344.32 (run as root)     [00:00] [PASS]
Test: /opt/libc-tests/tests/wcsncasecmp-7344.64 (run as root)     [00:00] [PASS]
Test: /opt/libc-tests/tests/wcsncasecmp-7350.32 (run as root)     [00:00] [PASS]
Test: /opt/libc-tests/tests/wcsncasecmp-7350.64 (run as root)     [00:00] [PASS]
Test: /opt/libc-tests/tests/wcsncasecmp.32 (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/wcsncasecmp.64 (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/wcsrtombs_test (run as root)          [00:00] [PASS]
Test: /opt/libc-tests/tests/wctype_test (run as root)             [00:00] [PASS]
Test: /opt/libc-tests/tests/symbols/setup (run as root)           [00:00] [PASS]
Test: /opt/libc-tests/tests/symbols/assert_h (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/symbols/ctype_h (run as root)         [00:24] [PASS]
Test: /opt/libc-tests/tests/symbols/dirent_h (run as root)        [00:05] [PASS]
Test: /opt/libc-tests/tests/symbols/fcntl_h (run as root)         [00:15] [PASS]
Test: /opt/libc-tests/tests/symbols/locale_h (run as root)        [00:11] [PASS]
Test: /opt/libc-tests/tests/symbols/math_h (run as root)          [00:04] [PASS]
Test: /opt/libc-tests/tests/symbols/netdb_h (run as root)         [00:01] [PASS]
Test: /opt/libc-tests/tests/symbols/pthread_h (run as root)       [00:02] [PASS]
Test: /opt/libc-tests/tests/symbols/signal_h (run as root)        [00:01] [PASS]
Test: /opt/libc-tests/tests/symbols/stdalign_h (run as root)      [00:02] [PASS]
Test: /opt/libc-tests/tests/symbols/stddef_h (run as root)        [00:00] [PASS]
Test: /opt/libc-tests/tests/symbols/stdio_h (run as root)         [00:14] [PASS]
Test: /opt/libc-tests/tests/symbols/stdlib_h (run as root)        [00:13] [PASS]
Test: /opt/libc-tests/tests/symbols/stdnoreturn_h (run as root)   [00:00] [PASS]
Test: /opt/libc-tests/tests/symbols/string_h (run as root)        [00:01] [PASS]
Test: /opt/libc-tests/tests/symbols/strings_h (run as root)       [00:00] [PASS]
Test: /opt/libc-tests/tests/symbols/sys_stat_h (run as root)      [00:07] [PASS]
Test: /opt/libc-tests/tests/symbols/sys_time_h (run as root)      [00:01] [PASS]
Test: /opt/libc-tests/tests/symbols/sys_timeb_h (run as root)     [00:00] [PASS]
Test: /opt/libc-tests/tests/symbols/time_h (run as root)          [00:01] [PASS]
Test: /opt/libc-tests/tests/symbols/threads_h (run as root)       [00:25] [PASS]
Test: /opt/libc-tests/tests/symbols/ucontext_h (run as root)      [00:02] [PASS]
Test: /opt/libc-tests/tests/symbols/unistd_h (run as root)        [00:19] [PASS]
Test: /opt/libc-tests/tests/symbols/wchar_h (run as root)         [00:10] [PASS]
Test: /opt/libc-tests/tests/symbols/wctype_h (run as root)        [00:24] [PASS]

Results Summary
PASS     107

Running Time:   00:03:31
Percent passed: 100.0%
Log directory:  /var/tmp/test_results/20200326T085624

And here's what a run of the individual memstream tests that I wrote looks like:

rm@nienor:/opt/libc-tests$ ./tests/stdio/memstream.32 
TEST PASSED: fmemopen: bad mode argument
TEST PASSED: fmemopen: bad buffer size, valid buf
TEST PASSED: fmemopen: bad buffer size, NULL buf
TEST PASSED: fmemopen: invalid NULL buf, mode: r
TEST PASSED: fmemopen: invalid NULL buf, mode: w
TEST PASSED: fmemopen: invalid NULL buf, mode: a
TEST PASSED: fmemopen: invalid NULL buf, mode: ax
TEST PASSED: fmemopen: bad open ask for SIZE_MAX bytes
TEST PASSED: fmemopen: simulate malloc failure at open
TEST PASSED: open_memstream: bad buf
TEST PASSED: open_memstream: bad size
TEST PASSED: open_memstream: bad buf and size
TEST PASSED: open_memstream: simulate malloc failure at open
TEST PASSED: open_wmemstream: bad buf
TEST PASSED: open_wmemstream: bad size
TEST PASSED: open_wmemstream: bad buf and size
TEST PASSED: open_wmemstream: simulate malloc failure at open
TEST PASSED: fmemopen: write beyond end of buffer: putc (buf smaller than BUFSIZ)
TEST PASSED: fmemopen: write beyond end of buffer: putc (line buffering)
TEST PASSED: fmemopen: write beyond end of buffer: putc (no stdio buffering)
TEST PASSED: fmemopen: write beyond end of buffer: fwrite (buf smaller than BUFSIZ)
TEST PASSED: fmemopen: write beyond end of buffer: fwrite (line buffering)
TEST PASSED: fmemopen: write beyond end of buffer: fwrite (no stdio buffering)
TEST PASSED: fmemopen: write beyond end of buffer: fwrite 2 (buf smaller than BUFSIZ)
TEST PASSED: fmemopen: write beyond end of buffer: fwrite 2 (line buffering)
TEST PASSED: fmemopen: write beyond end of buffer: fwrite 2 (no stdio buffering)
TEST PASSED: fmemopen: write beyond end of buffer: fputs (buf smaller than BUFSIZ)
TEST PASSED: fmemopen: write beyond end of buffer: fputs (line buffering)
TEST PASSED: fmemopen: write beyond end of buffer: fputs (no stdio buffering)
TEST PASSED: fmemopen: default position and log. size, mode: r
TEST PASSED: fmemopen: default position and log. size, mode: r+
TEST PASSED: fmemopen: default position and log. size, mode: w
TEST PASSED: fmemopen: default position and log. size, mode: w+
TEST PASSED: fmemopen: default position and log. size, mode: a
TEST PASSED: fmemopen: default position and log. size, mode: a+
TEST PASSED: fmemopen: default position and log. size, mode: a, nul byte
TEST PASSED: fmemopen: default position and log. size, mode: a+, nul byte
TEST PASSED: fmemopen: default position and log. size, mode: a+, NULL buf
TEST PASSED: fmemopen: read until EOF with fgetc
TEST PASSED: fmemopen: read until EOF with fgets
TEST PASSED: fmemopen: read until EOF with fread
TEST PASSED: fmemopen: read until EOF with fread 2
TEST PASSED: fmemopen: invalid seeks
TEST PASSED: fmemopen: w+ mode truncates buffer
TEST PASSED: fmemopen: NULs properly inserted (w)
TEST PASSED: fmemopen: NULs properly inserted (a)
TEST PASSED: fmemopen: read NUL character normally
TEST PASSED: open_memstream: default position and logical size
TEST PASSED: wopen_memstream: default position and logical size
TEST PASSED: open_memstream: read doesn't work
TEST PASSED: open_wmemstream: read doesn't work
TEST PASSED: open_memstream: flush failure due to induced memory failure
TEST PASSED: open_wmemstream: flush failure due to induced memory failure
TEST PASSED: open_[w]memstream: bad seeks
TEST PASSED: open_memstream: appends NULs
TEST PASSED: open_wmemstream: appends NULs
TEST PASSED: open_wmemstream: handles embedded NULs
TEST PASSED: open_wmemstream: write wide chars
TEST PASSED: open_wmemstream: seeking doesn't grow
TEST PASSED: open_wmemstream: Write mb sequences
TEST PASSED: open_wmemstream: detect bad utf-8 sequence
TEST PASSED: open_wmemstream: detect bad utf-8 sequence 2 (fflush)
TEST PASSED: open_wmemstream: ftell buffering behavior
63/63 tests passed