Project

General

Profile

Bug #115

/opt/onbld/bin/nightly[2377]: wstypes: line 2346: http_get: not found

Added by Garrett D'Amore over 10 years ago. Updated over 10 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Start date:
2010-08-29
Due date:
% Done:

100%

Estimated time:
Difficulty:
Tags:
Gerrit CR:

Description

When building illumos, the log will contain a message like this:

/opt/onbld/bin/nightly2377: wstypes: line 2346: http_get: not found

The problem is that http_get is defined in nightly after the first call to it. So the definition needs to be moved.

#1

Updated by Rich Lowe over 10 years ago

nightly has been this way for a very long time, (r7544 commit:7ecb3a3bcf8f).

I'm convinced the problem is that people are running a script which will be truncated as it's running, when nightly clobbers.

I've updated [[How To Build illumos]] to stop recommending people do this, and I would encourage someone to follow it, and see if they still see failures.
(that is, copy nightly someplace safe, then run it).

Really, after the initial bootstrap build and onu, people should be using pkg:/developer/build/onbld, and not needing this dance.

#2

Updated by Rich Lowe over 10 years ago

Some detective work suggests this is two bugs.

A user who was victim to this did as I asked and copied the script, and saw the error, but not the failure, so it appears the failure is nightly getting clobbered.

Now if we look at the definition of wstypes:


function wstypes {
    typeset parent_type child_type junk

    CODEMGR_WS="$BRINGOVER_WS" "$WHICH_SCM" 2>/dev/null \
        | read parent_type junk
    if [[ -z "$parent_type" || "$parent_type" == unknown ]]; then
        # Probe BRINGOVER_WS to determine its type
        if [[ $BRINGOVER_WS == svn*://* ]]; then
            parent_type="subversion" 
        elif [[ $BRINGOVER_WS == file://* ]] &&
            egrep -s "This is a Subversion repository" \
            ${BRINGOVER_WS#file://}/README.txt 2> /dev/null; then
            parent_type="subversion" 
        elif [[ $BRINGOVER_WS == ssh://* ]]; then
            parent_type="mercurial" 
        elif svn info $BRINGOVER_WS > /dev/null 2>&1; then
            parent_type="subversion" 
        elif [[ $BRINGOVER_WS == http://* ]] && \
            http_get "$BRINGOVER_WS/?cmd=heads" | \
            egrep -s "application/mercurial" 2> /dev/null; then
            parent_type="mercurial" 
        else
            parent_type="none" 
        fi
    fi

We see that it only does this stuff is which_scm can't be found, which makes sense, as we (SCM) wrote this code to let people cross the SCM boundary cleanly. For most of us, this doesn't fail, we have which_scm! However, that first time, when you don't have developer/build/onbld, it'll bomb, or if which_scm loses, it'll bomb.

Now, there's logic in nightly which tries to deal with this:


nightly_path=`whence $0`

WHICH_SCM=$(dirname $nightly_path)/which_scm
if [[ ! -x $WHICH_SCM ]]; then
    WHICH_SCM=which_scm
fi

But `whence $0` will be empty!

As a cheesy example:
richlowe@metropolis:~$ whence /home/richlowe/foo.ksh
richlowe@metropolis:~$ echo $?
1

So we don't find it anyway, despite our best attempts, and fall back to the wstypes logic which, yes, uses http_get too soon, but nobody noticed, because for most of us, it's never used at all.

#3

Updated by Rich Lowe over 10 years ago

  • Status changed from New to In Progress
  • Assignee set to Rich Lowe

Some detective work suggests this is two bugs.

A user who was victim to this did as I asked and copied the script, and saw the error, but not the failure, so it appears the failure is nightly getting clobbered.

Now if we look at the definition of wstypes:


function wstypes {
    typeset parent_type child_type junk

    CODEMGR_WS="$BRINGOVER_WS" "$WHICH_SCM" 2>/dev/null \
        | read parent_type junk
    if [[ -z "$parent_type" || "$parent_type" == unknown ]]; then
        # Probe BRINGOVER_WS to determine its type
        if [[ $BRINGOVER_WS == svn*://* ]]; then
            parent_type="subversion" 
        elif [[ $BRINGOVER_WS == file://* ]] &&
            egrep -s "This is a Subversion repository" \
            ${BRINGOVER_WS#file://}/README.txt 2> /dev/null; then
            parent_type="subversion" 
        elif [[ $BRINGOVER_WS == ssh://* ]]; then
            parent_type="mercurial" 
        elif svn info $BRINGOVER_WS > /dev/null 2>&1; then
            parent_type="subversion" 
        elif [[ $BRINGOVER_WS == http://* ]] && \
            http_get "$BRINGOVER_WS/?cmd=heads" | \
            egrep -s "application/mercurial" 2> /dev/null; then
            parent_type="mercurial" 
        else
            parent_type="none" 
        fi
    fi

We see that it only does this stuff is which_scm can't be found, which makes sense, as we (SCM) wrote this code to let people cross the SCM boundary cleanly. For most of us, this doesn't fail, we have which_scm! However, that first time, when you don't have developer/build/onbld, it'll bomb, or if which_scm loses, it'll bomb.

Now, there's logic in nightly which tries to deal with this:


nightly_path=`whence $0`

WHICH_SCM=$(dirname $nightly_path)/which_scm
if [[ ! -x $WHICH_SCM ]]; then
    WHICH_SCM=which_scm
fi

But `whence $0` will be empty!

As a cheesy example:
richlowe@metropolis:~$ whence /home/richlowe/foo.ksh
richlowe@metropolis:~$ echo $?
1

So we don't find it anyway, despite our best attempts, and fall back to the wstypes logic which, yes, uses http_get too soon, but nobody noticed, because for most of us, it's never used at all.

#4

Updated by Garrett D'Amore over 10 years ago

Um, I'm using a separate copy of nightly (specifically because I thought this might be the problem), made by copying nightly manually from the workspace into /opt/onbld/bin, and then running it from that location.

I still see the problem.

I don't understand why nobody has seen this problem before, but I'm quite confident that the problem is a bug in the script -- you can't call a function in the script before you have defined it. How this ever worked (or even if it worked but was not getting caught in the logs) is a mystery to me.

I'm less interested in history here... lets just fix it for the future, please.

#5

Updated by Joerg Goltermann over 10 years ago

At the begin of nightly.sh BRINGOVER_WS is defined from CLONE_WS

if [ "$BRINGOVER_WS" = "" ]; then
BRINGOVER_WS=$CLONE_WS
fi

CLONE_WS is defined: CLONE_WS="http://hg.illumos.org/illumos-gate" in illumos.sh

this will end in nightly.sh, function wstypes in the problem that http_get is not yet defined.

Opensolaris.sh defines CLONE_WS with CLONE_WS="ssh://hg/onnv/onnv-gate"

which will end in parent_type mercurial too, but without calling http_get

I have not checked what elese is done with CLONE_WS but I think it should be updated
to an other value like Opensolaris it does.

- Joerg

#6

Updated by Rich Lowe over 10 years ago

  • Subject changed from /opt/onbld/bin/nightly[2377]: wstypes: line 2346: http_get: not found to /opt/onbld/bin/nightly[2377]: wstypes: line 2346: http_get: not found
  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Fixed in r13161 commit:35133fa0d604

Also available in: Atom PDF