Feature #5788

Want support for GCC's stack protector in libc

Added by Andrew Stormont about 6 years ago. Updated 5 months ago.

lib - userland libraries
Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


Building code with the -fstack-protector option is more difficult than it needs to be. One way to make it easier would be to drop the need for libssp_nonshared.a by implementing __stack_chk_fail and __stack_chk_guard in libc.

Related issues

Related to illumos gate - Feature #5922: Want support for building with -fstack-protectorClosedAlex Wilson2015-05-13

Blocked by illumos gate - Feature #13273: want upanic(2)ClosedRobert Mustacchi

Blocks illumos gate - Feature #13274: enable -fstack-protector-strong by default in user landClosedRobert Mustacchi


Updated by Andrew Stormont almost 6 years ago

  • Related to Feature #5922: Want support for building with -fstack-protector added

Updated by Dan McDonald almost 6 years ago

Interesting. I've started using gcc51 in OmniOS's bloody for building non-ON bits. Does libc's lack of support here cause a problem?

Downloading any recent (as of this comment) OmniOS bloody should have gcc51-built omnios-build userland bits in it. I'd appreciate knowing if there are any interaction problems.


Updated by Andrew Stormont almost 6 years ago

It kind of does cause problems. If you pass -fstack-protector when compiling libssp gets linked in automatically, but if you are building objects separately and then do the final link with ld libssp does not get linked in, and you get unresolved symbol errors. The solution is to make sure that -lssp is passed when linking with ld. Linux (and BSD, afaik) include SSP support directly in libc so there's no need to futz with the separate libssp library.


Updated by Dan McDonald almost 6 years ago

The "on by default in newer gcc" from a similar bug had me concerned. I see no libssp linkages in omnios-build built tools, so perhaps 5.1.0 doesn't add -fstack-protector by default.


Updated by Andrew Stormont almost 6 years ago

I haven't seen it either. It must be disabled on Solaris/illumos, probably due to the issues I mentioned above.


Updated by Andrew Stormont over 5 years ago

I suppose adding some filter symbols to libc should be good enough.

EDIT: Never mind. That won't help.


Updated by Robert Mustacchi 6 months ago

  • Category set to lib - userland libraries
  • Assignee set to Robert Mustacchi
  • % Done changed from 0 to 80
  • Tags deleted (needs-triage)

Updated by Robert Mustacchi 6 months ago


Updated by Robert Mustacchi 6 months ago

To support the stack protector there are a few things we need:

1. libc needs the __stack_chk_fail function
2. libc needs the __stack_chk_guard pointer that is initialized with random data early on in program lifetime (more on how we do that later)
3. i386 needs the ability to link in the __stack_chk_fail_local function which is what was traditionally delivered by libssp_nonshared.a
4. The compiler needs to be updated to know about number 3.

What we have put together implements both 1 and 2 in libc. We use getrandom(2) to get random data for guard early on in the program. If that fails we fallback to a variant of the GCC stackguard paper's recommendation of the canary suggested by Alex Wilson. We use the '\0' and '\n' bytes, but also use the low two bytes of gethrtime() as a way to have some poor-quality randomness in there that the attacker can't readily guess, but still have the terminator canaries that are more likely to stop a wayward string function.

One difference from gcc, though more like musl, is that we do not attempt to try and print out anything to stderr when this occurs. We consider the process to be in a compromised state (and honestly it's hard to know that we'll be able to honestly call into libc) and so just call upanic(2).

For issue number three we are implementing an archive library that we're calling libssp_ns.a. This purposefully has a different name so that folks who have the existing gcc-based stack protector libraries installed do not have a conflict in any way. This will allow the illumos/gcc spec files to be updated to point to this to make this a first class citizen in those environments, though it also allows one the ability to link with -lssp_ns manually as we are doing in #13274 to avoid the compiler flag day.


Updated by Robert Mustacchi 6 months ago

  • Blocks Feature #13274: enable -fstack-protector-strong by default in user land added

Updated by Electric Monk 6 months ago

  • Gerrit CR set to 1040

Updated by Robert Mustacchi 5 months ago

To test the stack protector we did a few things:

  • We wrote small test programs that would trigger it on different heuristics and verified that we triggered the stack protector and the upanic(2) debugging support.
  • I went back and undid the ix for #13242 and verified that upon replicating the issue, that the stack protector would have caught it and that we properly aborted the process.
  • When running with #13274 we were able to catch a small issue in libvarpd/kernel interfaces and verified that this fired and worked in a multi-threaded daemon.
  • Andy wrote up patches for gcc and verified that everything worked end to end there.

Updated by Electric Monk 5 months ago

  • Status changed from New to Closed
  • % Done changed from 80 to 100

git commit 6a817834d81cc75ce12d0d393320837b1fec1e85

commit  6a817834d81cc75ce12d0d393320837b1fec1e85
Author: Robert Mustacchi <>
Date:   2020-11-17T16:52:10.000Z

    5788 Want support for GCC's stack protector in libc
    Reviewed by: Andy Fiddaman <>
    Reviewed by: Toomas Soome <>
    Approved by: Gordon Ross <>

Also available in: Atom PDF