About OpenSSL integration into illumos

Review Request #12 - Created March 21, 2015 and updated

Information
Gordon Ross
illumos-gate
Reviewers
general

About OpenSSL integration into illumos

We'd like an in-gate OpenSSL for illumos, in hopes of continuing to
reduce dependencies on external libraries etc. It is a goal of the
illumos project to eliminate dependencies on external libraries.

It was previously suggested (by Keith W.) that we integrate OpenSSL
into illumos in a way that only meets the needs of in-gate consumers,
and stays out of the way of an external OpenSSL that will typically be
provided by each distribution. That would mean the in-gate OpenSSL
would need to avoid exposing any commands, libraries, or even symbols
that might conflict with an "external" OpenSSL.

That approach does have its merits, and as Keith described at quite
some length, that's really the only way to go if your distro. has a
requirement to allow end users to choose their own OpenSSL.
On the other hand, if your distro has no interest in letting users
select their own OpenSSL implementation (i.e. an appliance) and
you'd rather have only one copy of this code to maintain, then the
approach Keith laid out doesn't really help you.

I had some other concerns as well about the approach that appeared
to be well underway, that would integrate a copy of about 90% of the
OpenSSL code into the gate. Historically, any time code has been
copied into the gate like that, it ended up changing in ways that
made it hard to update from a new upstream version, leading to a
stagnant copy of code that's difficult to update. I wanted a
solution that would let us build in-gate, with some flexibility
about build options etc, but also wanted to avoid the problems
associated with actually checking in a copy of the code.

I decided to investigate an approach that is conceptually sort of
inbetween the two extreems of "in" (copying the code into illumos)
and "out" (building the code out in the userland gate, or similar).
It's sort of a hybrid of the two. In this approach, I download
(if necessary) the source tar ball, unpack it, and apply patches
just like the userland gate does, but the similarity stops there.
I don't run configure and don't build the code using any of the
Makefiles provided. Instead, a collection of Makefiles under
usr/src/lib/openssl builds this code using pattern rules that
"reach over" to the unpacked/patched code.

There are several advantages to this approach:

  • We can directly use the patches from the userland gate.
    (That's what I did here, for the most part.)

  • It's fairly easy to update to a new version of OpenSSL
    (The last patch I present updates to 1.0.1m)

  • We can build both "sunw_" prefix libraries (as Keith wanted)
    and normal libraries. The Makefiles in usr/src/lib/openssl
    build both. Which you deliver is a packaging choice.
    Which you use in build is a configuration choice.

  • A new environment variable: SUNW_OPENSSL allows control
    over which openssl libraries the rest of the gate uses,
    either the "sunw_" prefix ones, or the normal ones.

Now if you're only interested in Keith's approach, you
won't care about these, but if you want to build all of
the OpenSSL deliverables, this work also lets you:

  • Build the rest of the OpenSSL "apps" (commands etc.)
    (See new Makefiles under usr/src/cmd/openssl/apps)

  • Build the tests (validation of libssl, libcrypto).
    (See new Makefiles under usr/src/cmd/openssl/test)

  • Build the man pages.
    (See new Makefiles under usr/src/man/man?openssl)

  • Build packages:
    normal libs, cmds etc. > library/security/openssl
    "sunw_" prefix libs > library/security/openssl/sunw
    headers, lint libs > library/security/openssl/dev
    man pages > library/security/openssl/man

Now for the code. The review is split into several parts:

1427 import openssl (Gary's $SRC/lib/openssl Makefiles etc)
https://www.illumos.org/rb/r/7/

1427 import openssl (introduce external/openssl/...)
https://www.illumos.org/rb/r/6/

1427 import libssl (build openssl libs from external/...)
https://www.illumos.org/rb/r/8/

1427 import libssl (build openssl cmd and tests)
https://www.illumos.org/rb/r/9/

1427 import libssl (man pages, packaging)
https://www.illumos.org/rb/r/10/

1427 import libssl (sunw_prefix stuff, lint)
https://www.illumos.org/rb/r/11/

1427 Update to OpenSSL 1.0.1m
https://www.illumos.org/rb/r/19/

If you want to look at the whole thing, see:
https://github.com/gwr/illumos-gate/commits/openssl-part1


  
Gordon Ross
Gordon Ross
Gordon Ross
Review request changed

Description:

   

About OpenSSL integration into illumos

   
   

We'd like an in-gate OpenSSL for illumos, in hopes of continuing to

    reduce dependencies on external libraries etc. It is a goal of the
    illumos project to eliminate dependencies on external libraries.

   
   

It was previously suggested (by Keith W.) that we integrate OpenSSL

    into illumos in a way that only meets the needs of in-gate consumers,
    and stays out of the way of an external OpenSSL that will typically be
    provided by each distribution. That would mean the in-gate OpenSSL
    would need to avoid exposing any commands, libraries, or even symbols
    that might conflict with an "external" OpenSSL.

   
   

That approach does have its merits, and as Keith described at quite

    some length, that's really the only way to go if your distro. has a
    requirement to allow end users to choose their own OpenSSL.
    On the other hand, if your distro has no interest in letting users
    select their own OpenSSL implementation (i.e. an appliance) and
    you'd rather have only one copy of this code to maintain, then the
    approach Keith laid out doesn't really help you.

   
   

I had some other concerns as well about the approach that appeared

    to be well underway, that would integrate a copy of about 90% of the
    OpenSSL code into the gate. Historically, any time code has been
    copied into the gate like that, it ended up changing in ways that
    made it hard to update from a new upstream version, leading to a
    stagnant copy of code that's difficult to update. I wanted a
    solution that would let us build in-gate, with some flexibility
    about build options etc, but also wanted to avoid the problems
    associated with actually checking in a copy of the code.

   
   

I decided to investigate an approach that is conceptually sort of

    inbetween the two extreems of "in" (copying the code into illumos)
    and "out" (building the code out in the userland gate, or similar).
    It's sort of a hybrid of the two. In this approach, I download
    (if necessary) the source tar ball, unpack it, and apply patches
    just like the userland gate does, but the similarity stops there.
    I don't run configure and don't build the code using any of the
    Makefiles provided. Instead, a collection of Makefiles under
    usr/src/lib/openssl builds this code using pattern rules that
    "reach over" to the unpacked/patched code.

   
   

There are several advantages to this approach:

   
   
  • We can directly use the patches from the userland gate.
    (That's what I did here, for the most part.)

   
  • It's fairly easy to update to a new version of OpenSSL
    (The last patch I present updates to 1.0.1m)

   
  • We can build both "sunw_" prefix libraries (as Keith wanted)
    and normal libraries. The Makefiles in usr/src/lib/openssl
    build both. Which you deliver is a packaging choice.
    Which you use in build is a configuration choice.

   
  • A new environment variable: SUNW_OPENSSL allows control
    over which openssl libraries the rest of the gate uses,
    either the "sunw_" prefix ones, or the normal ones.

   
   

Now if you're only interested in Keith's approach, you

    won't care about these, but if you want to build all of
    the OpenSSL deliverables, this work also lets you:

   
   
  • Build the rest of the OpenSSL "apps" (commands etc.)
    (See new Makefiles under usr/src/cmd/openssl/apps)

   
  • Build the tests (validation of libssl, libcrypto).
    (See new Makefiles under usr/src/cmd/openssl/test)

   
  • Build the man pages.
    (See new Makefiles under usr/src/man/man?openssl)

   
  • Build packages:
    normal libs, cmds etc. > library/security/openssl
    "sunw_" prefix libs > library/security/openssl/sunw
    headers, lint libs > library/security/openssl/dev
    man pages > library/security/openssl/man

   
   

   
   

Now for the code. The review is split into several parts:

   
   

1427 import openssl (Gary's $SRC/lib/openssl Makefiles etc)

    https://www.illumos.org/rb/r/7/

   
   

1427 import openssl (introduce external/openssl/...)

    https://www.illumos.org/rb/r/6/

   
   

1427 import libssl (build openssl libs from external/...)

    https://www.illumos.org/rb/r/8/

   
   

1427 import libssl (build openssl cmd and tests)

    https://www.illumos.org/rb/r/9/

   
   

1427 import libssl (man pages, packaging)

    https://www.illumos.org/rb/r/10/

   
   

1427 import libssl (sunw_prefix stuff, lint)

    https://www.illumos.org/rb/r/11/

   
   

1427 Update to OpenSSL 1.0.1m

    https://www.illumos.org/rb/r/19/

  +
  +

If you want to look at the whole thing, see:

  + https://github.com/gwr/illumos-gate/commits/openssl-part1

Loading...