Project

General

Profile

Bug #10347

Feature #9979: Support python3 for in-gate tools

git-pbchk requires more python3 fixes

Added by Andy Fiddaman 10 months ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
tools - gate/build tools
Start date:
2019-02-06
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:

Description

When running under python3, pbchk can produce a python stacktrace when spawning an external process such as cstyle.

Traceback (most recent call last):
  File "/build/illumos-omnios/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/git-pbchk", line 389, in <module>
    main(os.path.basename(sys.argv[0]), sys.argv[1:])
  File "/build/illumos-omnios/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/git-pbchk", line 381, in main
    pbchk(git_root(), parent_branch, None)
  File "/build/illumos-omnios/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/git-pbchk", line 348, in pbchk
    run_checks(root, parent, cmds)
  File "/build/illumos-omnios/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/git-pbchk", line 319, in run_checks
    output=s)
  File "/build/illumos-omnios/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/git-pbchk", line 266, in cstyle
    check_continuation=True)
  File "/build/illumos-omnios/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/../lib/python3.5/onbld/Checks/CStyle.py", line 55, in cstyle
    ret, tmpfile = processcheck('cstyle', options, fh, output)
  File "/build/illumos-omnios/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/../lib/python3.5/onbld/Checks/ProcessCheck.py", line 63, in processcheck
    p.stdin.write(line)
TypeError: a bytes-like object is required, not 'str'

This is happening because git-pbchk is opening the file in non-binary mode (which in python 3 means as text with automatic character conversion from UTF-8 by default), but then attempting to feed this to a stdin pipe which needs bytes and not str.

The fix is for git-pbchk to open the file in binary mode when it is going to send the file handle over to be used on a pipe, which is the case for cstyle and manlint, and to make the backend processcheck function use a binary file handle to record the stdout.

However, since the manlint function in pbchk passes the same file handle to both manlint (which spawns an external perl process) and to spellcheck (which is internal in python), I have also modified spellcheck to do a binary->str conversion on lines read from the file handle which is in binary mode.

History

#1

Updated by Andy Fiddaman 10 months ago

  • Description updated (diff)
#3

Updated by Andy Fiddaman 10 months ago

  • Status changed from In Progress to Pending RTI
  • % Done changed from 90 to 100

Testing: Built and ran a `git-pbchk` against the last 100 commits in gate using both python2 and python3.

#4

Updated by Electric Monk 10 months ago

  • Status changed from Pending RTI to Closed

git commit a90997d2f0a442a8aa8a56cbbbbf577716a18742

commit  a90997d2f0a442a8aa8a56cbbbbf577716a18742
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2019-02-09T21:43:47.000Z

    10347 git-pbchk requires more python3 fixes
    Reviewed by: Sebastian Wiedenroth <sebastian.wiedenroth@skylime.net>
    Reviewed by: C Fraire <cfraire@me.com>
    Reviewed by: John Levon <john.levon@joyent.com>
    Approved by: Joshua M. Clulow <josh@sysmgr.org>

Also available in: Atom PDF