Project

General

Profile

Actions

Bug #13434

closed

sh: mishandles backslash as last character of a block of input

Added by Zack Weinberg 4 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

Some versions of Illumos /bin/sh read script files in 8192-byte chunks. If the last character of one of these chunks is a backslash, it may be mishandled.

The specific bug I know about is: If the last line of a here-document straddles a chunk boundary, and the last character of that line is not a backslash, but the last character of the first chunk is a backslash, and the first character of the second chunk is a dollar sign, then the here-document will be written out without a final newline. (I'm not sure whether the boundary has to fall within the last line of the here-document, or whether the first character of the second chunk has to be a dollar sign. I know \# doesn't work, though.)

Because this appears to be a problem with low-level input handling, it's possible that this is not the only way backslash at a chunk boundary can cause a shell script to malfunction.

The attached file t-001fa2.sh.gz is a self-contained reproducer for the here-document bug. If you uncompress it and run it with a shell that has this bug, it will print something like this:

cmp: EOF on /tmp/tmp.LCyimJxG8v after byte 1, in line 1
+ xxd /tmp/tmp.q5iPccsLOn
00000000: 240a                                     $.
+ xxd /tmp/tmp.LCyimJxG8v
00000000: 24                                       $
+ exit 1
+ rm -f /tmp/tmp.q5iPccsLOn /tmp/tmp.LCyimJxG8v

Note that the script does not have a #! line. Also, you need to have xxd for it to work as intended, but both invocations of this utility are after the critical offset in the file, so you can safely change them to any other hex-dump program you may happen to have around without breaking the test. I have also attached the program I used to generate the test script, which may be easier to tinker with. The name t-001fa2 refers to the number of padding characters in the script, before the actual code.

This bug was originally reported as a problem running the Autoconf test suite on OmniOS. See https://savannah.gnu.org/support/?110403 for further discussion.


Files

t-001fa2.sh.gz (177 Bytes) t-001fa2.sh.gz reproducer for bug Zack Weinberg, 2021-01-05 04:33 PM
test-gen.py (1 KB) test-gen.py program that generates variations on the reproducer Zack Weinberg, 2021-01-05 04:33 PM

Related issues

Related to illumos gate - Feature #13405: ksh93 update to 2012-08-01ClosedAndy Fiddaman

Actions
Actions #1

Updated by Andy Fiddaman 4 months ago

Actions #2

Updated by Andy Fiddaman 4 months ago

  • Status changed from New to In Progress
  • Assignee set to Andy Fiddaman
Actions #3

Updated by Andy Fiddaman 4 months ago

I've confirmed that this issue is resolved by the upgraded version of ksh that I am working on in #13405

% ksh ~/ksh/chunktest
cmp: EOF on /tmp/tmp_12.tl9
+ xxd /tmp/tmp_1u.end
00000000: 240a                                     $.
+ xxd /tmp/tmp_12.tl9
00000000: 24                                       $
+ exit 1
+ rm -f /tmp/tmp_1u.end /tmp/tmp_12.tl9

% LD_LIBRARY_PATH=lib:usr/lib usr/bin/i86/ksh ~/ksh/old/test
% LD_LIBRARY_PATH=lib/64:usr/lib/64 usr/bin/amd64/ksh ~/ksh/old/test
%
Actions #4

Updated by Electric Monk 4 months ago

  • Gerrit CR set to 1175
Actions #5

Updated by Electric Monk 4 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit b30d193948be5a7794d7ae3ba0ed9c2f72c88e0f

commit  b30d193948be5a7794d7ae3ba0ed9c2f72c88e0f
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2021-01-30T17:13:33.000Z

    13405 ksh93 update to 2012-08-01
    13434 sh: mishandles backslash as last character of a block of input
    11750 ksh mkdir builtin doesn't honor special file permissions
    9199 ksh93 builtin *grep -v mishandles blank lines, blows up libgcrypt-config
    6756 sh (and ksh) have issues with ${1+"$@"}
    6520 ksh: sleep could wait forever
    4860 ksh93: core in printf
    3791 /bin/sh's builtin 'rm' busted: 'rm -f' without arguments returns error
    1047 ksh overwrites child core files
    880 ksh93 coredumps on 'unset'
    499 "interrupted system call" when using "tee" builtin in ksh
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Reviewed by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
    Reviewed by: Dominik Hassler <hadfl@omnios.org>
    Approved by: Rich Lowe <richlowe@richlowe.net>

Actions

Also available in: Atom PDF