Project

General

Profile

Actions
Copy link

Bug #4149

closed

ksh head builtin does not like newlines

Added by Jonathan Perkin over 8 years ago. Updated about 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Andy Fiddaman
Category:
cmd - userland programs
Start date:
2013-09-19
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
1325

Description

Best served by an example:

# /bin/ksh
# printf "hello" >file
# type head
head is a shell builtin version of /usr/bin/head
# head -1 file
# /usr/bin/head file
hello# uname -v
joyent_20130111T180733Z
#

This breaks scripts which do e.g. kill `head -1 /some/pid/file` if the pidfile does not have a trailing newline, most notably SMF method scripts which are usually #!/sbin/sh

Actions
Copy link
 #1

Updated by Andy Fiddaman about 1 year ago

  • Category set to cmd - userland programs
  • Status changed from New to In Progress
  • Assignee set to Andy Fiddaman
  • Tags deleted (needs-triage)

The builtin tail command code is using libast's sfmove() function to copy input to stdout, and this function is designed to work with complete records:

     Sfoff_t sfmove(Sfio_t* fr, Sfio_t* fw, Sfoff_t n, int rsc)
       This function moves objects from input stream fr to output stream fw.
       sfmove() returns the number of objects moved or -1 on failure.

       An object can be either a byte if the record separator argument rsc is
       negative or a record of rsc is non-negative.  In the latter case, a
       record is incomplete if it does not end in rsc.  Generally speaking, a
       stream can have at most one incomplete record.  If n is negative, all
       complete objects of fr will be moved.  Otherwise, n indicates the
       number of objects to move.  If either fr or fw is NULL, it acts as if
       it is a stream corresponding to /dev/null, the UNIX device that has no
       read data and throws away any write data.  For example, the call
       sfmove(f,(Sfio_t*)0,(Sfoff_t)(-1),'\n') counts the number of complete
       lines in stream f.

There is a patch in ksh93 2014-06-25 which resolves this by updating tail so that if it gets fewer records than expected (by the -n argument) then it copies the remaining bytes to the output. A small change to sfmove() was also required so that it used the bytes that were present in its reserve buffer.

Actions
Copy link
 #2

Updated by Electric Monk about 1 year ago

  • Gerrit CR set to 1325
Actions
Copy link
 #3

Updated by Andy Fiddaman about 1 year ago

The fix that I've put for review basically amounts to a backport of the fix from ksh93 2013:

13-09-19 head.c,tail.c: count incomplete lines too

I've separately added and run the new test which fails before this patch:

        illumos_4149_builtin_head.sh[82]: Shell head -1 one-line file
        illumos_4149_builtin_head.sh[85]: Shell head -2 one-line file
        illumos_4149_builtin_head.sh[108]: Shell head -6 six-line file
        illumos_4149_builtin_head.sh[111]: Shell head -10 six-line file
        illumos_4149_builtin_head.sh[114]: Shell tail -1 six-line file
        illumos_4149_builtin_head.sh[117]: Shell tail -2 six-line file

and succeeds after.

The other existing ksh93 tests produce the same results before and after this change.

Actions
Copy link
 #4

Updated by Electric Monk about 1 year ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit d6f391ef39bc41c64e16ac5d7b10c1c8d5b1761e

commit  d6f391ef39bc41c64e16ac5d7b10c1c8d5b1761e
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2021-03-16T19:20:39.000Z

    4149 ksh head builtin does not like newlines
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Actions
Copy link

Also available in: Atom PDF