Project

General

Profile

Actions

Bug #14514

closed

libfakekernel uses system headers, not proto or usr/src ones

Added by Dan McDonald 6 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
lib - userland libraries
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

With the recent use of upanic and <upanic.h> in libfakekernel, it has discovered both a bug in SmartOS (we did not ship /usr/include/upanic.h) and that libfakekernel uses system header files when it should probably use the ones in the gate it belongs in.

I added "-I$(SRC)/head. I tested this by using the -E of cw/cc to generate two evidence files, one as-is, and one with the aforementioned addition:

nowhere(~/build/illumos-gate)[0]% grep upanic.h /tmp/evidence
# 1 "/usr/include/upanic.h" 1 3 4
# 24 "/usr/include/upanic.h" 3 4
# 25 "/usr/include/upanic.h" 2 3 4
# 30 "/usr/include/upanic.h" 
# 30 "/usr/include/upanic.h" 3 4
nowhere(~/build/illumos-gate)[0]% grep upanic.h /tmp/evidence2
# 1 "/export/home/danmcd/build/illumos-gate/usr/src/head/upanic.h" 1
# 24 "/export/home/danmcd/build/illumos-gate/usr/src/head/upanic.h" 
# 25 "/export/home/danmcd/build/illumos-gate/usr/src/head/upanic.h" 2
nowhere(~/build/illumos-gate)[0]% 

I will link the SmartOS issue this uncovered later.


Related issues

Related to illumos gate - Bug #14479: SMB testoplock broken after 13515ClosedGordon Ross

Actions
Actions #1

Updated by Dan McDonald 6 months ago

This bug uncovered THIS bug in SmartOS: https://smartos.org/bugview/OS-8355

Actions #2

Updated by Dan McDonald 6 months ago

  • Related to Bug #14479: SMB testoplock broken after 13515 added
Actions #3

Updated by Dan McDonald 6 months ago

Tested by building illumos-joyent with libfakernel's #14479 merged in. It failed because of SmartOS OS-8355. With this fix in place:

smartos-build-2021-4(projects/illumos)[0]% git diff
diff --git a/usr/src/lib/libfakekernel/Makefile.com b/usr/src/lib/libfakekernel/Makefile.com
index 036dc5e205..831593e2f2 100644
--- a/usr/src/lib/libfakekernel/Makefile.com
+++ b/usr/src/lib/libfakekernel/Makefile.com
@@ -61,7 +61,7 @@ CFLAGS +=     $(CCVERBOSE)
 CPPFLAGS.first += -I../common
 CPPFLAGS= $(CPPFLAGS.first)

-INCS += -I$(SRC)/uts/common
+INCS += -I$(SRC)/uts/common -I$(SRC)/head

 CPPFLAGS += $(INCS) -D_REENTRANT -D_FAKE_KERNEL
 CPPFLAGS += -D_FILE_OFFSET_BITS=64
smartos-build-2021-4(projects/illumos)[0]% 

SmartOS builds with 14479 merged in, in spite of OS-8355.

Actions #4

Updated by Electric Monk 6 months ago

  • Gerrit CR set to 2029
Actions #5

Updated by Dan McDonald 6 months ago

Modified with -E added, but here are the compiler invocations in libfakekernel before and after this fix.

Before:

WS-nowhere-WS(libfakekernel/amd64)[0]% /export/home/danmcd/build/illumos-gate/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --linker /export/home/danmcd/build/illumos-gate/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/ld --primary gcc7,/opt/gcc-7//bin/gcc,gnu --shadow gcc10,/opt/gcc-10/bin/gcc,gnu --shadow smatch,/export/home/danmcd/build/illumos-gate/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/smatch,smatch -- -O -m32 -fpic -xspace -Xa -xildoff -errtags=yes -errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT -erroff=E_STATEMENT_NOT_REACHED -_gcc=-Wno-missing-braces -_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas -_gcc=-Wno-unused-parameter -_gcc=-Wno-missing-field-initializers -_gcc=-Wno-array-bounds -_smatch=-p=illumos_user -_smatch=--disable=uninitialized,check_check_deref -_smatch=-Wno-vla -_smatch=-Wno-one-bit-signed-bitfield -_smatch=-Wno-external-function-has-definition -_smatch=-Wno-old-style-definition -_smatch=-Wno-strict-prototypes -_smatch=--fatal-checks -_smatch=--timeout=0 -xc99=%all -W0,-xglobalstatic -_gcc=-fno-inline-small-functions -_gcc=-fno-inline-functions-called-once -_gcc=-fno-ipa-cp -_gcc=-fno-ipa-icf -_gcc=-fno-clone-functions -_gcc=-fno-reorder-functions -_gcc=-fno-reorder-blocks-and-partition -_gcc=-fno-aggressive-loop-optimizations -_gcc=--param=max-inline-insns-single=450 -_gcc=-fstack-protector-strong -v -g -_cc=-xdebugformat=stabs -_gcc=-gdwarf-2 -_gcc10=-gstrict-dwarf -_gcc11=-gstrict-dwarf -xc99=%all -W0,-noglobal -I../common -I/export/home/danmcd/build/illumos-gate/usr/src/uts/common -D_REENTRANT -D_FAKE_KERNEL -D_FILE_OFFSET_BITS=64 -DDEBUG -DPIC -D_REENTRANT -c -E ../common/printf.c > /tmp/evidence

After:

WS-nowhere-WS(libfakekernel/amd64)[0]% /export/home/danmcd/build/illumos-gate/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --linker /export/home/danmcd/build/illumos-gate/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/ld --primary gcc7,/opt/gcc-7//bin/gcc,gnu --shadow gcc10,/opt/gcc-10/bin/gcc,gnu --shadow smatch,/export/home/danmcd/build/illumos-gate/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/smatch,smatch -- -O -m32 -fpic -xspace -Xa -xildoff -errtags=yes -errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT -erroff=E_STATEMENT_NOT_REACHED -_gcc=-Wno-missing-braces -_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas -_gcc=-Wno-unused-parameter -_gcc=-Wno-missing-field-initializers -_gcc=-Wno-array-bounds -_smatch=-p=illumos_user -_smatch=--disable=uninitialized,check_check_deref -_smatch=-Wno-vla -_smatch=-Wno-one-bit-signed-bitfield -_smatch=-Wno-external-function-has-definition -_smatch=-Wno-old-style-definition -_smatch=-Wno-strict-prototypes -_smatch=--fatal-checks -_smatch=--timeout=0 -xc99=%all -W0,-xglobalstatic -_gcc=-fno-inline-small-functions -_gcc=-fno-inline-functions-called-once -_gcc=-fno-ipa-cp -_gcc=-fno-ipa-icf -_gcc=-fno-clone-functions -_gcc=-fno-reorder-functions -_gcc=-fno-reorder-blocks-and-partition -_gcc=-fno-aggressive-loop-optimizations -_gcc=--param=max-inline-insns-single=450 -_gcc=-fstack-protector-strong -v -g -_cc=-xdebugformat=stabs -_gcc=-gdwarf-2 -_gcc10=-gstrict-dwarf -_gcc11=-gstrict-dwarf -xc99=%all -W0,-noglobal -I../common -I/export/home/danmcd/build/illumos-gate/usr/src/uts/common -D_REENTRANT -D_FAKE_KERNEL -D_FILE_OFFSET_BITS=64 -DDEBUG -DPIC -D_REENTRANT -c -E ../common/printf.c > /tmp/evidence2
Actions #6

Updated by Dan McDonald 6 months ago

Testing done: in addition to SmartOS builds, I mangled /usr/include/upanic.h's name on purpose and built -gate on my OmniOS build zone:

nowhere(~/build)[0]% ls /usr/include/upanic*
/usr/include/upanic.h.notnow  
nowhere(~/build)[0]% fgbuild illumos-gate-gcc10.env
Using env file /export/home/danmcd/build/illumos-gate-gcc10.env for fgbuild
Waiting 45 seconds to start monitoring nightly.
CODEMGR_WS is /export/home/danmcd/build/illumos-gate
==== Nightly distributed build started:   Wed Feb 16 10:10:24 EST 2022 ====
==== list of environment variables ====
==== Nightly argument issues ====
==== Build version ====
==== Make clobber at Wed Feb 16 10:10:24 EST 2022 ====
==== Make tools clobber at Wed Feb 16 10:10:42 EST 2022 ====
==== No bringover to /export/home/danmcd/build/illumos-gate ====
==== Bootstrapping tools at Wed Feb 16 10:10:44 EST 2022 ====
==== Build environment ====
==== Building tools at Wed Feb 16 10:10:51 EST 2022 ====
==== New environment settings. ====
==== Building OS-Net source at Wed Feb 16 10:11:10 EST 2022 (non-DEBUG) ====
==== Ended OS-Net source build at Wed Feb 16 10:31:15 EST 2022 (non-DEBUG) ====
==== Not creating non-DEBUG packages ====
==== Building OS-Net source at Wed Feb 16 10:31:16 EST 2022 (DEBUG) ====
==== Ended OS-Net source build at Wed Feb 16 10:42:23 EST 2022 (DEBUG) ====
==== Not creating DEBUG packages ====
==== Find core files ====
==== Check lists of files ====
==== Nightly distributed build completed: Wed Feb 16 10:43:13 EST 2022 ====
==== Total build time ====
pwait: no such process: 685
nowhere(~/build)[0]% cd illumos-gate
nowhere(~/build/illumos-gate)[0]% git log -1
commit 94afd1448ba04525848cf1165d8deec88a124035 (HEAD -> master)
Author: Gordon Ross <gwr@racktopsystems.com>
Date:   Wed Feb 16 07:45:21 2022 -0500

    14479 SMB testoplock broken after 13515 (fix check_rtime)
nowhere(~/build/illumos-gate)[0]% git pull --all
Fetching origin
remote: Enumerating objects: 13, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 7 (delta 5), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), 595 bytes | 297.00 KiB/s, done.
From /export/home/danmcd/ws/illumos-gate
 + 679822776a...80411341a7 master     -> origin/master  (forced update)
Updating 94afd1448b..80411341a7
Fast-forward
 usr/src/lib/libfakekernel/Makefile.com | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
nowhere(~/build/illumos-gate)[0]% git log -1
commit 80411341a7fd366a46a5de9b758184c68fd63722 (HEAD -> master, origin/master, origin/HEAD)
Author: Dan McDonald <danmcd@joyent.com>
Date:   Wed Feb 16 10:07:26 2022 -0500

    14514 libfakekernel uses system headers, not proto or usr/src ones

    Change-Id: I01b15c526fbdcaac2082e24175adfaea0c57d3bc
nowhere(~/build/illumos-gate)[0]% fgbuild ../illumos-gate-gcc10.env 
Using env file /export/home/danmcd/build/illumos-gate/../illumos-gate-gcc10.env for fgbuild
Waiting 45 seconds to start monitoring nightly.
CODEMGR_WS is /export/home/danmcd/build/illumos-gate
==== Nightly distributed build started:   Wed Feb 16 10:43:47 EST 2022 ====
==== list of environment variables ====
==== Nightly argument issues ====
==== Build version ====
==== Make clobber at Wed Feb 16 10:43:47 EST 2022 ====
==== Make tools clobber at Wed Feb 16 10:44:04 EST 2022 ====
==== No bringover to /export/home/danmcd/build/illumos-gate ====
==== Bootstrapping tools at Wed Feb 16 10:44:06 EST 2022 ====
==== Build environment ====
==== Building tools at Wed Feb 16 10:44:14 EST 2022 ====
==== New environment settings. ====
==== Building OS-Net source at Wed Feb 16 10:44:32 EST 2022 (non-DEBUG) ====
==== Ended OS-Net source build at Wed Feb 16 11:05:14 EST 2022 (non-DEBUG) ====
==== Creating non-DEBUG packages at Wed Feb 16 11:05:15 EST 2022 ====
==== Building OS-Net source at Wed Feb 16 11:07:02 EST 2022 (DEBUG) ====
==== Ended OS-Net source build at Wed Feb 16 11:18:21 EST 2022 (DEBUG) ====
==== Creating DEBUG packages at Wed Feb 16 11:18:22 EST 2022 ====
==== Linting packages ====
==== Creating protolist system file at Wed Feb 16 11:19:29 EST 2022 ====
==== protolist system file created at Wed Feb 16 11:19:29 EST 2022 ====
==== Check ELF runtime attributes ====
==== Diff ELF runtime attributes (since last build) ====
==== Find core files ====
==== Diff unreferenced files (since last build) ====
==== Check lists of files ====
==== Nightly distributed build completed: Wed Feb 16 11:22:50 EST 2022 ====
==== Total build time ====
pwait: no such process: 27735
nowhere(~/build/illumos-gate)[0]% ls /usr/include/upanic*
/usr/include/upanic.h.notnow  
nowhere(~/build/illumos-gate)[0]% 
Actions #7

Updated by Dan McDonald 6 months ago

Tested the -gate build again with the -I$(ROOT)/usr/include change instead, as well.

Actions #8

Updated by Gordon Ross 6 months ago

BTW, the reason this first comes up now is that libfakekernel mostly concerns itself with kernel headers.
The exception, as seen here with upanic.h is that it implements many of these kernel-like interfaes
on top of libc interfaces, for which we may need some user-level headers.

(Actually, I think upanic.h may be the first such it has needed.
Nope: We were getting umem.h from the build host.)

Actions #9

Updated by Dan McDonald 6 months ago

  • % Done changed from 0 to 80
Actions #10

Updated by Electric Monk 6 months ago

  • Status changed from New to Closed
  • % Done changed from 80 to 100

git commit 59b827862fcc03b4da50df402eeb6288a75ac015

commit  59b827862fcc03b4da50df402eeb6288a75ac015
Author: Dan McDonald <danmcd@joyent.com>
Date:   2022-02-18T19:20:43.000Z

    14514 libfakekernel uses system headers, not proto or usr/src ones
    Reviewed by: Gordon Ross <gwr@racktopsystems.com>
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Approved by: Robert Mustacchi <rm@fingolfin.org>

Actions

Also available in: Atom PDF