Bug #11552
closedWant a more modern nawk(1)
100%
Description
The current illumos nawk
is effectively a fork from the state of Brian Kernighan's nawk
in 1989. Sun then restyled the code, and made the occasional independent fix, but they never pulled in upstream changes again. Since the two source bases had diverged significantly, I tried to recreate the history as best as I could, and used that to inform several updates for SmartOS: a mostly non-functional change, a functional change, and one notable breaking change. I then followed up with two other changes (a fix to field splitting, and support for the RT variable). This update for illumos contains all of those changes.
Note that this change only catches nawk(1)
up to about late 2018, since I haven't had a chance to evaluate other recent changes yet.
Related issues
Updated by Cody Mello almost 3 years ago
To test these changes, I made sure that all of the updates pulled in had appropriate tests in the test suite. I also verified that (with the changes in #11551) the gate will be able to build with the new binary. I rebooted OmniOS onto the build, and it came up fine (aside from some dbus issues due to illumos upstream libc not having the inotify_*
functions that are in SmartOS and OmniOS).
Additionally, SmartOS has been running with an updated nawk
since release-20190620
, and we haven't yet seen or received reports of any issues.
Running the updated test suite against an updated nawk
binary:
cpm@utu ~ % cd ws/illumos-gate/proto/root_i386/opt/util-tests/tests/awk cpm@utu ...util-tests/tests/awk : master % ./runtests.sh -o ~/ws/illumos-gate/proto/root_i386/usr/bin/nawk Running AWK tests ($AWK="/home/cpm/ws/illumos-gate/proto/root_i386/usr/bin/nawk") # Examples from "The AWK Programming Environment" examples/awk/p.1... ok examples/awk/p.10... ok examples/awk/p.11... ok examples/awk/p.12... ok examples/awk/p.13... ok examples/awk/p.14... ok examples/awk/p.15... ok examples/awk/p.16... ok examples/awk/p.17... ok examples/awk/p.18... ok examples/awk/p.19... ok examples/awk/p.2... ok examples/awk/p.20... ok examples/awk/p.21... ok examples/awk/p.21a... ok examples/awk/p.22... ok examples/awk/p.23... ok examples/awk/p.24... ok examples/awk/p.25... ok examples/awk/p.26... ok examples/awk/p.26a... ok examples/awk/p.27... ok examples/awk/p.28... ok examples/awk/p.29... ok examples/awk/p.3... ok examples/awk/p.30... ok examples/awk/p.31... ok examples/awk/p.32... ok examples/awk/p.33... ok examples/awk/p.34... ok examples/awk/p.35... ok examples/awk/p.36... ok examples/awk/p.37... ok examples/awk/p.38... ok examples/awk/p.39... ok examples/awk/p.4... ok examples/awk/p.40... ok examples/awk/p.41... ok examples/awk/p.42... ok examples/awk/p.43... ok examples/awk/p.44... ok examples/awk/p.45... ok examples/awk/p.46... ok examples/awk/p.47... ok examples/awk/p.48... ok examples/awk/p.48a... ok examples/awk/p.48b... ok examples/awk/p.49... ok examples/awk/p.5... ok examples/awk/p.50... ok examples/awk/p.51... ok examples/awk/p.52... ok examples/awk/p.5a... ok examples/awk/p.6... ok examples/awk/p.7... ok examples/awk/p.8... ok examples/awk/p.9... ok examples/awk/p.table... ok # One True AWK Example Programs examples/awk/t.0... ok examples/awk/t.0a... ok examples/awk/t.1... ok examples/awk/t.1.x... ok examples/awk/t.2... ok examples/awk/t.2.x... ok examples/awk/t.3... ok examples/awk/t.3.x... ok examples/awk/t.4... ok examples/awk/t.4.x... ok examples/awk/t.5.x... ok examples/awk/t.6... ok examples/awk/t.6.x... ok examples/awk/t.6a... ok examples/awk/t.6b... ok examples/awk/t.8.x... ok examples/awk/t.8.y... ok examples/awk/t.addops... ok examples/awk/t.aeiou... ok examples/awk/t.aeiouy... ok examples/awk/t.arith... ok examples/awk/t.array... ok examples/awk/t.array1... ok examples/awk/t.array2... ok examples/awk/t.assert... ok examples/awk/t.avg... ok examples/awk/t.b.x... ok examples/awk/t.be... ok examples/awk/t.beginexit... ok examples/awk/t.beginnext... ok examples/awk/t.break... ok examples/awk/t.break1... ok examples/awk/t.break2... ok examples/awk/t.break3... ok examples/awk/t.bug1... ok examples/awk/t.builtins... ok examples/awk/t.cat... ok examples/awk/t.cat1... ok examples/awk/t.cat2... ok examples/awk/t.cmp... ok examples/awk/t.coerce... ok examples/awk/t.coerce2... ok examples/awk/t.comment... ok examples/awk/t.comment1... ok examples/awk/t.concat... ok examples/awk/t.cond... ok examples/awk/t.contin... ok examples/awk/t.count... ok examples/awk/t.crlf... ok examples/awk/t.cum... ok examples/awk/t.d.x... ok examples/awk/t.delete0... ok examples/awk/t.delete1... ok examples/awk/t.delete2... ok examples/awk/t.delete3... ok examples/awk/t.do... ok examples/awk/t.e... ok examples/awk/t.else... ok examples/awk/t.exit... ok examples/awk/t.exit1... ok examples/awk/t.f... ok examples/awk/t.f.x... ok examples/awk/t.f0... ok examples/awk/t.f1... ok examples/awk/t.f2... ok examples/awk/t.f3... ok examples/awk/t.f4... ok examples/awk/t.for... ok examples/awk/t.for1... ok examples/awk/t.for2... ok examples/awk/t.for3... ok examples/awk/t.format4... ok examples/awk/t.fun... ok examples/awk/t.fun0... ok examples/awk/t.fun1... ok examples/awk/t.fun2... ok examples/awk/t.fun3... ok examples/awk/t.fun4... ok examples/awk/t.fun5... ok examples/awk/t.getline1... ok examples/awk/t.getval... ok examples/awk/t.gsub... ok examples/awk/t.gsub1... ok examples/awk/t.gsub3... ok examples/awk/t.gsub4... ok examples/awk/t.i.x... ok examples/awk/t.if... ok examples/awk/t.in... ok examples/awk/t.in1... ok examples/awk/t.in2... ok examples/awk/t.in3... ok examples/awk/t.incr... ok examples/awk/t.incr2... ok examples/awk/t.incr3... ok examples/awk/t.index... ok examples/awk/t.intest... ok examples/awk/t.intest2... ok examples/awk/t.j.x... ok examples/awk/t.longstr... ok examples/awk/t.makef... ok examples/awk/t.match... ok examples/awk/t.match1... ok examples/awk/t.max... ok examples/awk/t.mod... ok examples/awk/t.monotone... ok examples/awk/t.nameval... ok examples/awk/t.next... ok examples/awk/t.NF... ok examples/awk/t.not... ok examples/awk/t.null0... ok examples/awk/t.ofmt... ok examples/awk/t.ofs... Warning: missing newline at end of file /tmp/doCa41e Warning: missing newline at end of file examples/out/t.ofs ok examples/awk/t.ors... Warning: missing newline at end of file /tmp/dsCai2e Warning: missing newline at end of file examples/out/t.ors ok examples/awk/t.pat... ok examples/awk/t.pipe... ok examples/awk/t.pp... ok examples/awk/t.pp1... ok examples/awk/t.pp2... ok examples/awk/t.printf... ok examples/awk/t.printf2... ok examples/awk/t.quote... ok examples/awk/t.randk... ok examples/awk/t.re1... ok examples/awk/t.re1a... ok examples/awk/t.re2... ok examples/awk/t.re3... ok examples/awk/t.re4... ok examples/awk/t.re5... ok examples/awk/t.re7... ok examples/awk/t.rec... ok examples/awk/t.redir1... ok examples/awk/t.reFS... ok examples/awk/t.reg... ok examples/awk/t.roff... ok examples/awk/t.sep... ok examples/awk/t.seqno... ok examples/awk/t.set0... ok examples/awk/t.set0a... ok examples/awk/t.set0b... ok examples/awk/t.set1... ok examples/awk/t.set2... ok examples/awk/t.set3... ok examples/awk/t.split1... ok examples/awk/t.split2... ok examples/awk/t.split2a... ok examples/awk/t.split3... ok examples/awk/t.split4... ok examples/awk/t.split8... ok examples/awk/t.split9... ok examples/awk/t.split9a... ok examples/awk/t.stately... ok examples/awk/t.strcmp... ok examples/awk/t.strcmp1... ok examples/awk/t.strnum... ok examples/awk/t.sub0... ok examples/awk/t.sub1... ok examples/awk/t.sub2... ok examples/awk/t.sub3... ok examples/awk/t.substr... ok examples/awk/t.substr1... ok examples/awk/t.time... ok examples/awk/t.vf... ok examples/awk/t.vf1... ok examples/awk/t.vf2... ok examples/awk/t.vf3... ok examples/awk/t.x... ok a-format.awk... ok concat-assign-same.awk... ok decr-NF.awk... ok fs-overflow.awk... ok negative-nf.awk... ok nf-self-assign.awk... ok numeric-fs.awk... ok numeric-output-seps.awk... Warning: missing newline at end of file /tmp/nawktest.1dt341/test.temp.out Warning: missing newline at end of file /tmp/deMaOgf ok numeric-rs.awk... ok numeric-subsep.awk... ok ofs-rebuild.awk... ok space.awk... ok split-fs-from-array.awk... ok string-conv.awk... ok subsep-overflow.awk... ok system-status.awk... ok unary-plus.awk... ok # One True AWK Test Programs ./T.-f-f... ok ./T.argv... ok ./T.builtin... ok ./T.chem... ok ./T.close... ok ./T.clv... ok ./T.delete... ok ./T.errmsg... ok ./T.expr... ok ./T.exprconv... ok ./T.flags... ok ./T.func... ok ./T.gawk... ok ./T.getline... ok ./T.latin1... ok ./T.lilly... ok ./T.main... ok ./T.misc... ok ./T.nextfile... ok ./T.overflow... ok ./T.re... ok ./T.recache... ok ./T.redir... ok ./T.rt... ok ./T.split... ok ./T.sub... ok ./T.system... ok # Imported GAWK Test Programs anchgsub.awk... ok anchor.awk... ok arrayind3.awk... ok arrayprm2.awk... ok arrayprm3.awk... ok arrayref.awk... ok arrymem1.awk... ok arynasty.awk... ok arynocls.awk... ok aryprm9.awk... ok arysubnm.awk... ok aryunasgn.awk... ok asgext.awk... ok back89.awk... ok childin.awk... ok clobber.awk... ok closebad.awk... ok clsflnam.awk... ok compare2.awk... ok concat1.awk... ok concat2.awk... ok concat3.awk... ok concat4.awk... ok concat5.awk... ok convfmt.awk... ok datanonl.awk... ok delarprm.awk... ok dfastress.awk... ok dynlj.awk... ok eofsplit.awk... ok exit2.awk... ok exitval2.awk... ok exitval3.awk... ok fcall_exit.awk... ok fcall_exit2.awk... ok fldchg.awk... ok fldchgnf.awk... ok fldterm.awk... ok fordel.awk... ok forsimp.awk... ok fsbs.awk... ok fsrs.awk... ok fsspcoln.awk... ok fstabplus.awk... ok funsemnl.awk... ok funstack.awk... ok gawksub.awk... ok getline3.awk... ok getlnbuf.awk... ok getlnhd.awk... ok getnr2tb.awk... ok getnr2tm.awk... ok gsubtest.awk... ok gsubtst2.awk... ok gsubtst5.awk... ok gsubtst7.awk... ok gsubtst8.awk... ok hex.awk... ok hex2.awk... ok inpref.awk... ok inputred.awk... ok intest.awk... ok intprec.awk... ok iobug1.awk... ok leadnl.awk... ok longsub.awk... ok longwrds.awk... ok manglprm.awk... ok math.awk... ok membug1.awk... ok minusstr.awk... ok mtchi18n.awk... ok nasty.awk... ok nasty2.awk... ok negexp.awk... ok negrange.awk... ok nested.awk... ok nfldstr.awk... ok nfloop.awk... ok nlfldsep.awk... ok nlinstr.awk... ok nlstrina.awk... ok noloop1.awk... ok noloop2.awk... ok nulrsend.awk... ok numindex.awk... ok numstr1.awk... ok numsubstr.awk... ok octsub.awk... ok ofmt.awk... ok ofmta.awk... ok ofmtbig.awk... ok ofmtfidl.awk... ok ofmts.awk... ok ofmtstrnum.awk... ok onlynl.awk... ok opasnidx.awk... ok opasnslf.awk... ok paramtyp.awk... ok paramuninitglobal.awk... ok parse1.awk... ok pcntplus.awk... ok pipeio1.awk... ok pipeio2.awk... ok prdupval.awk... ok prec.awk... ok printf0.awk... ok printf1.awk... ok printfchar.awk... ok prmreuse.awk... ok prt1eval.awk... ok range1.awk... ok rebrackloc.awk... ok rebt8b1.awk... ok rebuild.awk... ok redfilnm.awk... ok regeq.awk... ok regexpbrack2.awk... ok regexprange.awk... ok reparse.awk... ok resplit.awk... ok rri1.awk... ok rs.awk... ok rscompat.awk... ok rsnul1nl.awk... ok rstest2.awk... ok rstest3.awk... ok rstest4.awk... ok rstest5.awk... ok rstest6.awk... ok rswhite.awk... ok setrec0.awk... ok setrec1.awk... ok sigpipe1.awk... ok splitargv.awk... ok splitarr.awk... ok splitdef.awk... ok splitvar.awk... ok splitwht.awk... ok status-close.awk... ok strcat1.awk... ok strnum1.awk... ok strnum2.awk... ok strtod.awk... ok subamp.awk... ok subi18n.awk... ok subsepnm.awk... ok subslash.awk... ok substr.awk... ok swaplns.awk... ok tweakfld.awk... ok uninit2.awk... ok uninit5.awk... ok uparrfs.awk... ok wideidx.awk... ok wideidx2.awk... ok widesub.awk... ok widesub2.awk... ok widesub3.awk... ok wjposer1.awk... ok zero2.awk... ok zeroe0.awk... ok zeroflag.awk... ok ./compare.sh... ok ./fflush.sh... ok ./getline2.sh... ok ./leaddig.sh... ok ./localenl.sh... ok ./messages.sh... ok ./nors.sh... ok ./poundbang.sh... ok ./randtest.sh... ok ./rsnulbig.sh... ok ./rsnulbig2.sh... ok # Imported GAWK Syntax Tests arrayparm.awk... ok arryref3.awk... ok arryref4.awk... ok arryref5.awk... ok aryprm1.awk... ok aryprm2.awk... ok aryprm3.awk... ok aryprm4.awk... ok aryprm5.awk... ok aryprm6.awk... ok badassign1.awk... ok badbuild.awk... ok callparam.awk... ok delfunc.awk... ok fnarray.awk... ok fnaryscl.awk... ok fnmisc.awk... ok funsmnam.awk... ok gsubasgn.awk... ok nastyparm.awk... ok nfneg.awk... ok noparms.awk... ok nulinsrc.awk... ok paramdup.awk... ok parseme.awk... ok readbuf.awk... ok scalar.awk... ok sclforin.awk... ok sclifin.awk... ok synerr1.awk... ok synerr2.awk... ok tradanch1.awk... ok unterm.awk... ok TOTAL: 475/475
Updated by Cody Mello almost 3 years ago
Copying all of the util-tests
and the test runner into /opt
, all of the tests pass fine, including the AWK tests:
cpm@utu /opt/util-tests % export PATH=/opt/onbld/bin:/opt/onbld/bin/i386/:$PATH cpm@utu /opt/util-tests % pfexec ./bin/utiltest -c ./runfiles/default.run Test: /opt/util-tests/tests/allowed-ips (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/chown_test (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/date_test (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/grep_test (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libjedec_test (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libsff/libsff (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/mergeq/mqt (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/mergeq/wqt (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/printf_test (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/set-linkprop (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/smbios (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/xargs_test (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/awk/runtests.sh (run as nobody) [01:53] [PASS] Test: /opt/util-tests/tests/ctf/precheck (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/ctf/ctftest (run as root) [00:03] [PASS] Test: /opt/util-tests/tests/demangle/afl-fast (run as root) [00:01] [PASS] Test: /opt/util-tests/tests/demangle/gcc-libstdc++ (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/demangle/llvm-stdcxxabi (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_00_blank (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_01_boolean (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_02_numbers (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_03_empty_arrays (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_04_number_arrays (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_05_strings (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_06_nested (run as root) [00:00] [PASS] Test: /opt/util-tests/tests/libnvpair_json/json_07_nested_arrays (run as root) [00:00] [PASS] Results Summary PASS 26 Running Time: 00:02:01 Percent passed: 100.0% Log directory: /var/tmp/test_results/20190821T154809
Updated by Electric Monk almost 3 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit 3ee4fc2aa6b5136515cc3eed32d3c6ef33e37471
commit 3ee4fc2aa6b5136515cc3eed32d3c6ef33e37471 Author: Cody Peter Mello <cody.mello@joyent.com> Date: 2019-08-24T02:54:33.000Z 11552 Want a more modern nawk(1) Reviewed by: Robert Mustacchi <rm@joyent.com> Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com> Reviewed by: Andy Fiddaman <andy@omniosce.org> Approved by: Gordon Ross <gwr@nexenta.com>
Updated by Robert Mustacchi over 2 years ago
- Related to Feature #3731: Update nawk to version 20121220 added