Actions
Bug #14418
closed
mman.h symbol visibility is a mess
Start date:
Due date:
% Done:
100%
Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
External Bug:
Description
Right now mman.,h has many different contoritions for getting access to symbols in its #ifdef's. However, critically, even if you ask for __EXTENSIONS__ and a bunch of XPG standards, which should really be expected these days, there is no way to get access to a number of the symbols such as getpagesizes or memcntl. In addition, there are old versions of several of the functions that take caddr_t arguments rather than the standardized void *.
Now, mman.h has a bit of history due to the fact that it contains both things that were visible in POSIX.1/POSIX.2 before the merge with the eventual single unix spec. The following symbols were present in SUS V4.2:
- mmap
- munmap
- mprotect
- msync
The following symbols were all added in SUSv5 and previously were part of the POSIX realtime specs:
- mlock
- munlock
- mlockall
- munlockall
- shm_open
- shm_unlink
SUSv6 / POSIX added a single function, posix_memalign.
Finally, we have a bunch of functions that are extensions that we've added over the decades:
- mincore
- memcntl
- getpagesizes
- getpagesizes2
- mmapobj
- meminfo
In this change, we opt to standardize each of these groups. In particular we now make every block in the vein of strict symbols, were appropriate. The following table summarizes what we've done:
| Type | .Pre-Processor |
| Our Extensions | !defined(STRICT_SYMBOLS) |
| mmap group | Nothing -- always visible |
| mlock group | !defined(STRICT_SYMBOLS) OR defined(POSIX_C_SOURCE) OR defined(XPG_5) |
| posix_memalign | !defined(_STRICT_SYMBOLS) OR defined(_XPG6) |
It's worth a bit of history and explanation with these. The easy ones here for the moment are posix_memalign, because it's a simple XPG6 addition. Our own extensions have used various awkward symbol gates; however, the relevant things for us is basically no strict symbols being defined. We've opted for this as a simple way to get the no-posix or extensions defined. The mmap group was always defined in illumos headers, regardless of whether standards were present. The major difference there was whether or not the functions described took a caddr_t or a void *. As part of this clean up, we opt to only expose the void * to software, hoping this'll result in a more portable world. Finally the mlock group was in the realtime posix group which requires POSIX_C_SOURCE > 2 and weren't in XPG until v5, so hence we have that morass.
This is still a work in progress so the above is subject to change.
Related issues
Updated by Robert Mustacchi almost 2 years ago
A first pass set of tests was a new suite of symbol visibility tests for this:
rm@beowulf:~$ pfexec /opt/libc-tests/bin/libctest Test: /opt/libc-tests/tests/aligned_alloc.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/aligned_alloc.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/c11_threads.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/c11_threads.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/c11_tss.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/c11_tss.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/call_once.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/call_once.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/catopen (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/endian.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/endian.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/env-7076.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/env-7076.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/fnmatch.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/fnmatch.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/fpround_test (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/i18n/bindtextdomain_test (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/memchr.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/memchr.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/memset_s.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/memset_s.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/newlocale_test (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/nl_langinfo_test (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/posix_memalign.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/posix_memalign.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/printf-6961.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/printf-9511.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/printf-9511.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/priv_gettext (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/psignal (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/pthread_attr_get_np (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/qsort/qsort_test (run as root) [00:02] [PASS] Test: /opt/libc-tests/tests/quick_exit (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4key.ksh (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4random (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4random_fork (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4random_forkall (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4random_forksig (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4random_prefork (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4random_preforkall (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/arc4random_preforksig (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/chacha (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/getentropy (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/getrandom (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/inz_child (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/inz_inval (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/inz_mlock (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/inz_region (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/inz_split (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/inz_split_vpp (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/random/inz_vpp (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/regex/regex_test (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/select/select.sh (run as root) [00:06] [PASS] Test: /opt/libc-tests/tests/set_constraint_handler_s.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/set_constraint_handler_s.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/fileno.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/fileno.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/fmemopentest.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/fmemopentest.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/ftell_ungetc.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/ftell_ungetc.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/ftello_12768.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/ftello_12768.lfs (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/memstream.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/memstream.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/memstream_reopen.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/memstream_reopen.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/open_memstreamtest.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/open_memstreamtest.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/orientation_test.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/orientation_test.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/test_mbrtowc.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/stdio/test_mbrtowc.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/strcoll-strxfrm-6907.32 (run as root) [00:01] [PASS] Test: /opt/libc-tests/tests/strcoll-strxfrm-6907.64 (run as root) [00:01] [PASS] Test: /opt/libc-tests/tests/strerror (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/thread_name (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/timespec_get.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/timespec_get.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/uchar.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/uchar.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/utimes.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/utimes.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/wcsncasecmp-7344.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/wcsncasecmp-7344.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/wcsncasecmp-7350.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/wcsncasecmp-7350.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/wcsncasecmp.32 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/wcsncasecmp.64 (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/wcsrtombs_test (run as root) [00:01] [PASS] Test: /opt/libc-tests/tests/wctype_test (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/symbols/setup (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/symbols/assert_h (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/symbols/ctype_h (run as root) [00:26] [PASS] Test: /opt/libc-tests/tests/symbols/dirent_h (run as root) [00:06] [PASS] Test: /opt/libc-tests/tests/symbols/fcntl_h (run as root) [00:16] [PASS] Test: /opt/libc-tests/tests/symbols/locale_h (run as root) [00:12] [PASS] Test: /opt/libc-tests/tests/symbols/math_h (run as root) [00:04] [PASS] Test: /opt/libc-tests/tests/symbols/netdb_h (run as root) [00:01] [PASS] Test: /opt/libc-tests/tests/symbols/pthread_h (run as root) [00:03] [PASS] Test: /opt/libc-tests/tests/symbols/signal_h (run as root) [00:01] [PASS] Test: /opt/libc-tests/tests/symbols/stdalign_h (run as root) [00:02] [PASS] Test: /opt/libc-tests/tests/symbols/stddef_h (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/symbols/stdio_h (run as root) [00:15] [PASS] Test: /opt/libc-tests/tests/symbols/stdlib_h (run as root) [00:14] [PASS] Test: /opt/libc-tests/tests/symbols/stdnoreturn_h (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/symbols/string_h (run as root) [00:02] [PASS] Test: /opt/libc-tests/tests/symbols/strings_h (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/symbols/sys_mman_h (run as root) [00:26] [PASS] Test: /opt/libc-tests/tests/symbols/sys_stat_h (run as root) [00:08] [PASS] Test: /opt/libc-tests/tests/symbols/sys_time_h (run as root) [00:03] [PASS] Test: /opt/libc-tests/tests/symbols/sys_timeb_h (run as root) [00:00] [PASS] Test: /opt/libc-tests/tests/symbols/time_h (run as root) [00:01] [PASS] Test: /opt/libc-tests/tests/symbols/threads_h (run as root) [00:27] [PASS] Test: /opt/libc-tests/tests/symbols/ucontext_h (run as root) [00:03] [PASS] Test: /opt/libc-tests/tests/symbols/unistd_h (run as root) [00:21] [PASS] Test: /opt/libc-tests/tests/symbols/wchar_h (run as root) [00:11] [PASS] Test: /opt/libc-tests/tests/symbols/wctype_h (run as root) [00:27] [PASS] Results Summary PASS 118 Running Time: 00:04:18 Percent passed: 100.0% Log directory: /var/tmp/test_results/20220118T174543
In parallel I'll be working with folks to get this reviewed and tested against larger package builds.
Updated by Robert Mustacchi almost 2 years ago
- Has duplicate Bug #14504: madvise() not declared if _XOPEN_SOURCE defined added
Updated by Robert Mustacchi over 1 year ago
In addition to the final versions of the symbol tests, with the help of Andy, Jonathan, and Rich I was able to verify:
- That we didn't regress any of the OmniOS core and extra packages
- That all of pkgsrc seemed fine in the end. The original pkgsrc bulk builds caught some of the issues around symbol visibility.
- Rich went through and verified that this fixed issues with a recent LLVM build
Updated by Electric Monk over 1 year ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit df5cd018c34371890eeeb8c930245b9323e8aa25
commit df5cd018c34371890eeeb8c930245b9323e8aa25
Author: Robert Mustacchi <rm@fingolfin.org>
Date: 2022-04-20T17:46:45.000Z
14418 mman.h symbol visibility is a mess
Reviewed by: Gergő Mihály Doma <domag02@gmail.com>
Reviewed by: Rich Lowe <richlowe@richlowe.net>
Approved by: Dan McDonald <danmcd@joyent.com>
Actions