Bug #4149


ksh head builtin does not like newlines

Added by Jonathan Perkin over 8 years ago. Updated about 1 year ago.

cmd - userland programs
Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


Best served by an example:

# /bin/ksh
# printf "hello" >file
# type head
head is a shell builtin version of /usr/bin/head
# head -1 file
# /usr/bin/head file
hello# uname -v

This breaks scripts which do e.g. kill `head -1 /some/pid/file` if the pidfile does not have a trailing newline, most notably SMF method scripts which are usually #!/sbin/sh

Actions #1

Updated by Andy Fiddaman about 1 year ago

  • Category set to cmd - userland programs
  • Status changed from New to In Progress
  • Assignee set to Andy Fiddaman
  • Tags deleted (needs-triage)

The builtin tail command code is using libast's sfmove() function to copy input to stdout, and this function is designed to work with complete records:

     Sfoff_t sfmove(Sfio_t* fr, Sfio_t* fw, Sfoff_t n, int rsc)
       This function moves objects from input stream fr to output stream fw.
       sfmove() returns the number of objects moved or -1 on failure.

       An object can be either a byte if the record separator argument rsc is
       negative or a record of rsc is non-negative.  In the latter case, a
       record is incomplete if it does not end in rsc.  Generally speaking, a
       stream can have at most one incomplete record.  If n is negative, all
       complete objects of fr will be moved.  Otherwise, n indicates the
       number of objects to move.  If either fr or fw is NULL, it acts as if
       it is a stream corresponding to /dev/null, the UNIX device that has no
       read data and throws away any write data.  For example, the call
       sfmove(f,(Sfio_t*)0,(Sfoff_t)(-1),'\n') counts the number of complete
       lines in stream f.

There is a patch in ksh93 2014-06-25 which resolves this by updating tail so that if it gets fewer records than expected (by the -n argument) then it copies the remaining bytes to the output. A small change to sfmove() was also required so that it used the bytes that were present in its reserve buffer.

Actions #2

Updated by Electric Monk about 1 year ago

  • Gerrit CR set to 1325
Actions #3

Updated by Andy Fiddaman about 1 year ago

The fix that I've put for review basically amounts to a backport of the fix from ksh93 2013:

13-09-19 head.c,tail.c: count incomplete lines too

I've separately added and run the new test which fails before this patch:[82]: Shell head -1 one-line file[85]: Shell head -2 one-line file[108]: Shell head -6 six-line file[111]: Shell head -10 six-line file[114]: Shell tail -1 six-line file[117]: Shell tail -2 six-line file

and succeeds after.

The other existing ksh93 tests produce the same results before and after this change.

Actions #4

Updated by Electric Monk about 1 year ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit d6f391ef39bc41c64e16ac5d7b10c1c8d5b1761e

commit  d6f391ef39bc41c64e16ac5d7b10c1c8d5b1761e
Author: Andy Fiddaman <>
Date:   2021-03-16T19:20:39.000Z

    4149 ksh head builtin does not like newlines
    Reviewed by: Toomas Soome <>
    Approved by: Dan McDonald <>


Also available in: Atom PDF