Project

General

Profile

Bug #10462

nightly errors - libmakestate.so.1: open failed

Added by Andy Fiddaman 8 months ago. Updated 8 months ago.

Status:
Closed
Priority:
High
Assignee:
Category:
tools - gate/build tools
Start date:
2019-02-26
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:

Description

Occasionally when running a full nightly build, the DEBUG portion of the build fails with the following error:

/code/10455/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/../../lib//i386/64/libmakestate.so.1: open failed: No such file or directory

This happens for me about 20% of the time. When I see the error, it is at the start of the DEBUG build stage.

For the usual full build process, nightly does some bootstrapping and builds the tools first using the host utilities and then it switches to using the tools just built and proceeds to run a non-DEBUG and DEBUG build in sequence.

The intention here is that the tools are built first and then used for the rest of the build. However, since some compiler flags change between the non-DEBUG and DEBUG build phases, make decides that it needs to rebuild the tools

% make -d all CERRWARN=
MAKEFLAGS value: e
    Building install.bin.o because new command
        /data/omnios-build/omniosorg/rti/ig_Illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --primary gcc4,/opt/gcc-4.4.4/bin/gcc,gnu ...
    different from old
        /data/omnios-build/omniosorg/rti/ig_Illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --primary gcc4,/opt/gcc-4.4.4/bin/gcc,gnu ...

This can be seen in the following extract from nightly.log, looking just at the build of install.bin.c - when the DEBUG phase starts, it is rebuilt due to the removal of the -_gcc=-Wno-unused -_gcc=-Wno-empty-body parameters in the DEBUG build.

==== Building tools at Tue Feb  5 10:43:08 UTC 2019 ====

/data/omnios-build/omniosorg/rti/ig_bhyve/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --primary gcc4,/opt/gcc-4.4.4/bin/gcc,gnu --shadow gcc7,/opt/gcc-7/bin/gcc,gnu -- -O -m32 -Xa  -xildoff -errtags=yes -errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT -erroff=E_STATEMENT_NOT_REACHED -_gcc=-Wno-missing-braces -_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas -_gcc=-Wno-unused-parameter -_gcc=-Wno-missing-field-initializers -_gcc=-Wno-array-bounds -_gcc=-Wno-unused -_gcc=-Wno-empty-body -_smatch=-p=illumos_user -_smatch=--disable=uninitialized,check_check_deref,unreachable -_smatch=-Wno-vla -_smatch=-Wno-one-bit-signed-bitfield -_smatch=-Wno-external-function-has-definition -_smatch=-Wno-old-style-definition -_smatch=-Wno-strict-prototypes   -_gcc=-Wno-type-limits -xc99=%none    -W0,-xglobalstatic -_gcc=-fno-inline-small-functions  -_gcc=-fno-inline-functions-called-once  -_gcc=-fno-ipa-cp  -_gcc7=-fno-ipa-icf  -_gcc8=-fno-ipa-icf  -_gcc7=-fno-clone-functions  -_gcc8=-fno-clone-functions  -_gcc7=-fno-reorder-functions  -_gcc8=-fno-reorder-functions   -v -D_TS_ERRNO -I../protocmp -c  install.bin.c

==== Building OS-Net source at Tue Feb  5 10:43:25 UTC 2019 (non-DEBUG) ====

/usr/bin/rm -f /data/omnios-build/omniosorg/rti/ig_bhyve/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install
/usr/bin/ln /data/omnios-build/omniosorg/rti/ig_bhyve/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install.bin /data/omnios-build/omniosorg/rti/ig_bhyve/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install

==== Building OS-Net source at Tue Feb  5 11:01:33 UTC 2019 (DEBUG) ====

/data/omnios-build/omniosorg/rti/ig_bhyve/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --primary gcc4,/opt/gcc-4.4.4/bin/gcc,gnu --shadow gcc7,/opt/gcc-7/bin/gcc,gnu -- -O -m32 -Xa  -xildoff -errtags=yes -errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT -erroff=E_STATEMENT_NOT_REACHED -_gcc=-Wno-missing-braces -_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas -_gcc=-Wno-unused-parameter -_gcc=-Wno-missing-field-initializers -_gcc=-Wno-array-bounds -_smatch=-p=illumos_user -_smatch=--disable=uninitialized,check_check_deref,unreachable -_smatch=-Wno-vla -_smatch=-Wno-one-bit-signed-bitfield -_smatch=-Wno-external-function-has-definition -_smatch=-Wno-old-style-definition -_smatch=-Wno-strict-prototypes   -_gcc=-Wno-type-limits -xc99=%none    -W0,-xglobalstatic -_gcc=-fno-inline-small-functions  -_gcc=-fno-inline-functions-called-once  -_gcc=-fno-ipa-cp  -_gcc7=-fno-ipa-icf  -_gcc8=-fno-ipa-icf  -_gcc7=-fno-clone-functions  -_gcc8=-fno-clone-functions  -_gcc7=-fno-reorder-functions  -_gcc8=-fno-reorder-functions   -v -D_TS_ERRNO -I../protocmp -c  install.bin.c

History

#1

Updated by Andy Fiddaman 8 months ago

Since nightly builds non-DEBUG tools and expects them to be used for the remainder of the build
(there's even a comment in there - "Tools should only be built non-DEBUG."),
I'm going to propose a patch to usr/src/Makefile that will always do a non-DEBUG build for the bldtools target.

With this change in place, nightly.log does not show the tools being rebuilt in the DEBUG phase:

% egrep '^===|install\.bin\>' log/latest/nightly.log
==== Nightly distributed build started:   Tue Feb 26 15:10:28 UTC 2019 ====
==== list of environment variables ====
==== Nightly argument issues ====
==== Build version ====
==== Make clobber at Tue Feb 26 15:10:28 UTC 2019 ====
/data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/install.bin
/usr/bin/rm -f install.bin.o stdusers.o
/usr/bin/rm -f install.bin install.bin.po
==== Make tools clobber at Tue Feb 26 15:10:53 UTC 2019 ====
/data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/install.bin
/usr/bin/rm -f install.bin.o stdusers.o
/usr/bin/rm -f install.bin install.bin.po
==== No bringover to /data/omnios-build/omniosorg/bloody/illumos ====
==== Bootstrapping cw ====
==== Build environment ====
==== Building tools at Tue Feb 26 15:10:55 UTC 2019 ====
/data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/install.bin
/data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --primary gcc4,/opt/gcc-4.4.4/bin/gcc,gnu --shadow gcc7,/opt/gcc-7/bin/gcc,gnu -- -O -m32 -Xa  -xildoff -errtags=yes -errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT -erroff=E_STATEMENT_NOT_REACHED -_gcc=-Wno-missing-braces -_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas -_gcc=-Wno-unused-parameter -_gcc=-Wno-missing-field-initializers -_gcc=-Wno-array-bounds -_gcc=-Wno-unused -_gcc=-Wno-empty-body -_smatch=-p=illumos_user -_smatch=--disable=uninitialized,check_check_deref,unreachable -_smatch=-Wno-vla -_smatch=-Wno-one-bit-signed-bitfield -_smatch=-Wno-external-function-has-definition -_smatch=-Wno-old-style-definition -_smatch=-Wno-strict-prototypes   -_gcc=-Wno-type-limits -xc99=%none    -W0,-xglobalstatic -_gcc=-fno-inline-small-functions  -_gcc=-fno-inline-functions-called-once  -_gcc=-fno-ipa-cp  -_gcc7=-fno-ipa-icf  -_gcc8=-fno-ipa-icf  -_gcc7=-fno-clone-functions  -_gcc8=-fno-clone-functions  -_gcc7=-fno-reorder-functions  -_gcc8=-fno-reorder-functions   -v -D_TS_ERRNO -I../protocmp -c  install.bin.c
+ /opt/gcc-4.4.4/bin/gcc -fident -finline -fno-inline-functions -fno-builtin -fno-asm -fdiagnostics-show-option -nodefaultlibs -D__sun -O -m32 -Wall -Wextra -Werror -Wno-missing-braces -Wno-sign-compare -Wno-unknown-pragmas -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -Wno-unused -Wno-empty-body -Wno-type-limits -std=gnu89 -fno-inline-small-functions -fno-inline-functions-called-once -fno-ipa-cp -D_TS_ERRNO -I../protocmp -c install.bin.c
+ /opt/gcc-7/bin/gcc -fident -finline -fno-inline-functions -fno-builtin -fno-asm -fdiagnostics-show-option -nodefaultlibs -D__sun -O -m32 -Wall -Wextra -Werror -Wno-missing-braces -Wno-sign-compare -Wno-unknown-pragmas -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -Wno-unused -Wno-empty-body -Wno-type-limits -std=gnu89 -fno-inline-small-functions -fno-inline-functions-called-once -fno-ipa-cp -fno-ipa-icf -fno-clone-functions -fno-reorder-functions -D_TS_ERRNO -I../protocmp -c install.bin.c -o /tmp/nightly.tmpdir.10870/cw.bIaGwg/cwcIaWwg.o
/data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw --primary gcc4,/opt/gcc-4.4.4/bin/gcc,gnu --shadow gcc7,/opt/gcc-7/bin/gcc,gnu -- -O -m32 -Xa  -xildoff -errtags=yes -errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT -erroff=E_STATEMENT_NOT_REACHED -_gcc=-Wno-missing-braces -_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas -_gcc=-Wno-unused-parameter -_gcc=-Wno-missing-field-initializers -_gcc=-Wno-array-bounds -_gcc=-Wno-unused -_gcc=-Wno-empty-body -_smatch=-p=illumos_user -_smatch=--disable=uninitialized,check_check_deref,unreachable -_smatch=-Wno-vla -_smatch=-Wno-one-bit-signed-bitfield -_smatch=-Wno-external-function-has-definition -_smatch=-Wno-old-style-definition -_smatch=-Wno-strict-prototypes   -_gcc=-Wno-type-limits -xc99=%none    -W0,-xglobalstatic -_gcc=-fno-inline-small-functions  -_gcc=-fno-inline-functions-called-once  -_gcc=-fno-ipa-cp  -_gcc7=-fno-ipa-icf  -_gcc8=-fno-ipa-icf  -_gcc7=-fno-clone-functions  -_gcc8=-fno-clone-functions  -_gcc7=-fno-reorder-functions  -_gcc8=-fno-reorder-functions   -v -D_TS_ERRNO -I../protocmp -M/data/omnios-build/omniosorg/bloody/illumos/usr/src/common/mapfiles/common/map.noexstk -M/data/omnios-build/omniosorg/bloody/illumos/usr/src/common/mapfiles/common/map.noexdata  -M/data/omnios-build/omniosorg/bloody/illumos/usr/src/common/mapfiles/common/map.pagealign -o install.bin install.bin.o stdusers.o -lgen
+ /opt/gcc-4.4.4/bin/gcc -fident -finline -fno-inline-functions -fno-builtin -fno-asm -fdiagnostics-show-option -nodefaultlibs -D__sun -O -m32 -Wall -Wextra -Werror -Wno-missing-braces -Wno-sign-compare -Wno-unknown-pragmas -Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds -Wno-unused -Wno-empty-body -Wno-type-limits -std=gnu89 -fno-inline-small-functions -fno-inline-functions-called-once -fno-ipa-cp -D_TS_ERRNO -I../protocmp -Wl,-M/data/omnios-build/omniosorg/bloody/illumos/usr/src/common/mapfiles/common/map.noexstk -Wl,-M/data/omnios-build/omniosorg/bloody/illumos/usr/src/common/mapfiles/common/map.noexdata -Wl,-M/data/omnios-build/omniosorg/bloody/illumos/usr/src/common/mapfiles/common/map.pagealign -o install.bin install.bin.o stdusers.o -lgen -lc
/usr/bin/rm -f /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install.bin; /usr/bin/cp -f install.bin /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386; /usr/bin/chmod 0555 /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install.bin
/usr/bin/ln /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install.bin /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install
==== New environment settings. ====
==== Building OS-Net source at Tue Feb 26 15:11:12 UTC 2019 (non-DEBUG) ====
/data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/install.bin
/usr/bin/ln /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install.bin /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install
==== Ended OS-Net source build at Tue Feb 26 15:32:22 UTC 2019 (non-DEBUG) ====
==== Creating non-DEBUG packages at Tue Feb 26 15:32:23 UTC 2019 ====
==== Building OS-Net source at Tue Feb 26 15:34:54 UTC 2019 (DEBUG) ====
/data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/install.bin
/usr/bin/ln /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install.bin /data/omnios-build/omniosorg/bloody/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/install
==== Ended OS-Net source build at Tue Feb 26 15:55:08 UTC 2019 (DEBUG) ====
==== Creating DEBUG packages at Tue Feb 26 15:55:09 UTC 2019 ====
==== Creating protolist system file at Tue Feb 26 15:57:38 UTC 2019 ====
==== protolist system file created at Tue Feb 26 15:57:38 UTC 2019 ====
==== Check versioning and ABI information ====
==== Check ELF runtime attributes ====
==== Diff ELF runtime attributes (since last build) ====
==== No 'dmake lint' ====
==== Find core files ====
==== Check lists of files ====
==== Nightly distributed build completed: Tue Feb 26 16:03:07 UTC 2019 ====
==== Total build time ====
#2

Updated by Andy Fiddaman 8 months ago

#3

Updated by Andy Fiddaman 8 months ago

  • Description updated (diff)
  • % Done changed from 0 to 90
#4

Updated by Andy Fiddaman 8 months ago

  • Status changed from In Progress to Pending RTI
  • % Done changed from 90 to 100
#5

Updated by Electric Monk 8 months ago

  • Status changed from Pending RTI to Closed

git commit 6d3b6de806befb1050c80e5d4be2214ee68fb2b7

commit  6d3b6de806befb1050c80e5d4be2214ee68fb2b7
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2019-03-01T10:09:12.000Z

    10462 nightly errors - libmakestate.so.1: open failed
    Reviewed by: Gergő Mihály Doma <domag02@gmail.com>
    Reviewed by: Jason King <jason.king@joyent.com>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF