Bug #4149
closedksh head builtin does not like newlines
100%
Description
Best served by an example:
# /bin/ksh # printf "hello" >file # type head head is a shell builtin version of /usr/bin/head # head -1 file # /usr/bin/head file hello# uname -v joyent_20130111T180733Z #
This breaks scripts which do e.g. kill `head -1 /some/pid/file`
if the pidfile does not have a trailing newline, most notably SMF method scripts which are usually #!/sbin/sh
Updated by Andy Fiddaman about 1 year ago
- Category set to cmd - userland programs
- Status changed from New to In Progress
- Assignee set to Andy Fiddaman
- Tags deleted (
needs-triage)
The builtin tail command code is using libast's sfmove()
function to copy input to stdout, and this function is designed to work with complete records:
Sfoff_t sfmove(Sfio_t* fr, Sfio_t* fw, Sfoff_t n, int rsc) This function moves objects from input stream fr to output stream fw. sfmove() returns the number of objects moved or -1 on failure. An object can be either a byte if the record separator argument rsc is negative or a record of rsc is non-negative. In the latter case, a record is incomplete if it does not end in rsc. Generally speaking, a stream can have at most one incomplete record. If n is negative, all complete objects of fr will be moved. Otherwise, n indicates the number of objects to move. If either fr or fw is NULL, it acts as if it is a stream corresponding to /dev/null, the UNIX device that has no read data and throws away any write data. For example, the call sfmove(f,(Sfio_t*)0,(Sfoff_t)(-1),'\n') counts the number of complete lines in stream f.
There is a patch in ksh93 2014-06-25 which resolves this by updating tail so that if it gets fewer records than expected (by the -n
argument) then it copies the remaining bytes to the output. A small change to sfmove()
was also required so that it used the bytes that were present in its reserve buffer.
Updated by Andy Fiddaman about 1 year ago
The fix that I've put for review basically amounts to a backport of the fix from ksh93 2013:
13-09-19 head.c,tail.c: count incomplete lines too
I've separately added and run the new test which fails before this patch:
illumos_4149_builtin_head.sh[82]: Shell head -1 one-line file illumos_4149_builtin_head.sh[85]: Shell head -2 one-line file illumos_4149_builtin_head.sh[108]: Shell head -6 six-line file illumos_4149_builtin_head.sh[111]: Shell head -10 six-line file illumos_4149_builtin_head.sh[114]: Shell tail -1 six-line file illumos_4149_builtin_head.sh[117]: Shell tail -2 six-line file
and succeeds after.
The other existing ksh93 tests produce the same results before and after this change.
Updated by Electric Monk about 1 year ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
git commit d6f391ef39bc41c64e16ac5d7b10c1c8d5b1761e
commit d6f391ef39bc41c64e16ac5d7b10c1c8d5b1761e Author: Andy Fiddaman <omnios@citrus-it.co.uk> Date: 2021-03-16T19:20:39.000Z 4149 ksh head builtin does not like newlines Reviewed by: Toomas Soome <tsoome@me.com> Approved by: Dan McDonald <danmcd@joyent.com>