Actions
Bug #10448
closedwsdiff explodes on encoding error
Start date:
2019-02-25
Due date:
% Done:
100%
Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
Description
With the recent updates to wsdiff
, it seems we've regressed on the ability to correctly diff certain files:
18:34:51 + /opt/onbld/bin/wsdiff -r artefacts/wsdiff.txt previous/proto/root_i386 current/proto/root_i386 18:37:48 platform/i86xpv/kernel/amd64/unix 18:38:27 platform/i86pc/kernel/amd64/unix 18:41:54 etc/motd 18:45:40 usr/sbin/iasl 18:46:14 usr/sbin/acpixtract 18:46:15 usr/sbin/acpidump 19:38:36 usr/lib/spell/hlistb 19:38:36 Exception in thread Thread-2: 19:38:36 Traceback (most recent call last): 19:38:36 File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner 19:38:36 self.run() 19:38:36 File "/opt/onbld/bin/wsdiff", line 1233, in run 19:38:36 compareOneFile(base, ptch, False) 19:38:36 File "/opt/onbld/bin/wsdiff", line 1277, in compareOneFile 19:38:36 return compareBasic(base, ptch, quiet, fileType) 19:38:36 File "/opt/onbld/bin/wsdiff", line 1112, in compareBasic 19:38:36 difference(fileName, fileType, diffs) 19:38:36 File "/opt/onbld/bin/wsdiff", line 195, in difference 19:38:36 log_difference(f, dtype, diffs) 19:38:36 File "/opt/onbld/bin/wsdiff", line 212, in log_difference 19:38:36 print(diffs[:diffs_sz_thresh], file=log) 19:38:36 UnicodeEncodeError: 'ascii' codec can't encode characters in position 64-66: ordinal not in range(128)
Note that usr/lib/spell/hlistb
, a binary data file, differs by two bytes between the two workspaces:
root@jenkins-oi0:~# diff \ <(LANG=C od -t x1 /export/jenkins/workspace/wsdiff/previous/proto/root_i386/usr/lib/spell/hlistb) \ <(LANG=C od -t x1 /export/jenkins/workspace/wsdiff/current/proto/root_i386/usr/lib/spell/hlistb) 416c416 < 0014760 d7 f1 1a 93 1a d8 7e e8 98 bf ff a9 7c de 89 b1 --- > 0014760 d7 f1 1a 93 1a d8 7e e8 98 7f ce a9 7c de 89 b1
Richard Lowe and I have previously noticed that hlistb
(and hlista
) occasionally differ by one or two bytes for no reason we've yet been able to determine -- but we noticed this because prior to the changes, wsdiff
did correctly render the differences.
Related issues
Updated by Joshua M. Clulow about 3 years ago
- Related to Bug #10225: wsdiff broken after 9979 added
Updated by Joshua M. Clulow about 3 years ago
- Related to Feature #9979: Support python3 for in-gate tools added
Updated by Joshua M. Clulow about 3 years ago
- Related to Bug #10449: wsdiff hangs forever after an exception is thrown added
Updated by Joshua M. Clulow about 3 years ago
Note that the error seen above occurs when wsdiff
is run in the en_US.UTF-8
locale. If I instead run the program with LANG=C LC_ALL=C
, it doesn't blow up. It's not clear that the diff is correct, though:
usr/lib/spell/hlistb NOTE: ASCII difference detected. 523,3661c523,3661 < 0010120 032 032 ~ | e 022 V 005 ~ ~ s $ r s $ 6 < 7c7e1a1a 05561265 24737e7e 36247372 < 0010140 e u * 032 M 3 q $ 7 025 N | 002 P L 037 < 1a2a7565 2471334d 7c4e1537 1f4c5002 < 0010160 K { A S # Q r 1 - E ~ \0 \0 \0 \0 ' < 53417b4b 31725123 007e452d 27000000 < 0010200 " S B i * 020 K 030 _ g N # 006 \b 033 < 42532220 4b102a69 4e675f18 1b080623 < 0010220 ) c c w ] ^ 027 9 \ t { k Y Z \b q < 77636329 39175e5d 6b7b745c 71085a59 < 0010240 \n 007 @ 002 K X n \b S 016 025 \0 007 1 M 0 ...
Updated by Andy Fiddaman about 3 years ago
- Status changed from New to In Progress
- Assignee set to Andy Fiddaman
Updated by Electric Monk about 3 years ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
git commit 2f7dba3e6747cbaaf1deb86e6ca1e2a5c96332ac
commit 2f7dba3e6747cbaaf1deb86e6ca1e2a5c96332ac Author: Andy Fiddaman <omnios@citrus-it.co.uk> Date: 2019-03-14T20:01:47.000Z 10524 wsdiff much slower after move from deprecated commands module 10448 wsdiff explodes on encoding error 10525 wsdiff output is not correct for a binary file 10526 wsdiff tries to spawn 4.8 threads Reviewed by: Gergő Mihály Doma <domag02@gmail.com> Reviewed by: Sebastian Wiedenroth <sebastian.wiedenroth@skylime.net> Approved by: Rich Lowe <richlowe@richlowe.net>
Actions