Project

General

Profile

Bug #12254

python changes have broken the zfs test suite

Added by Jerry Jelinek 10 months ago. Updated 8 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

After some recent python-related changes, the zfs test suite now fails with:

Traceback (most recent call last):
  File "/opt/test-runner/bin/run", line 913, in <module>
    main()
  File "/opt/test-runner/bin/run", line 907, in main
    testrun.run(options)
  File "/opt/test-runner/bin/run", line 740, in run
    self.testgroups[testgroup].run(self.logger, options)
  File "/opt/test-runner/bin/run", line 495, in run
    test.run(options)
  File "/opt/test-runner/bin/run", line 264, in run
    self.result.stdout, self.result.stderr = self.collect_output(proc)
  File "/opt/test-runner/bin/run", line 228, in collect_output
    fd.read()
  File "/opt/test-runner/bin/run", line 137, in read
    while self._read() is not None:
  File "/opt/test-runner/bin/run", line 149, in _read
    buf = os.read(fd, 4096).decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 24: ordinal not in range(128)

After putting in a workaround for that failure, the next failure happens here:

Traceback (most recent call last):
  File "/opt/test-runner/bin/run", line 913, in <module>
    main()
  File "/opt/test-runner/bin/run", line 907, in main
    testrun.run(options)
  File "/opt/test-runner/bin/run", line 740, in run
    self.testgroups[testgroup].run(self.logger, options)
  File "/opt/test-runner/bin/run", line 499, in run
    test.log(logger, options)
  File "/opt/test-runner/bin/run", line 321, in log
    out.write('%s\n' % line)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 24-25: ordinal not in range(128)

I am not sure if this effects other tests suites aside from zfs-test.


Related issues

Related to illumos gate - Bug #12397: 12254 broke the zfs test suite for older python versionsNew

Actions
#1

Updated by Jerry Jelinek 10 months ago

I've used the following changes in /opt/test-runner/bin/run to workaround these failures:

149c149
<         buf = os.read(fd, 4096).decode()
---
>         buf = os.read(fd, 4096).decode(encoding = 'UTF-8', errors = 'replace')
320a321
>                     line = line.encode('ascii', 'replace')
324a326
>                     line = line.encode('ascii', 'replace')
328a331
>                     line = line.encode('ascii', 'replace')

Since I am not a python programmer, I am not sure if this is the best fix.

#2

Updated by Alexander Pyhalov 10 months ago

Can we just ensure that it always runs in UTF-8 locale?

#3

Updated by Alexander Pyhalov 10 months ago

I've created and tested the following patch: http://buildzone.oi-build.r61.net/webrev-12254/ .

I've verified that functional/bootfs tests don't longer fail. Checked that util-tests pass. Also checked another zfs tests. Fixed issue found while running channel programs tests. One channel program test still fails here by unrelated issue -
/opt/zfs-tests/tests/functional/channel_program/synctask_core/tst.terminate_by_signal fails with:

11:01:36.17 NOTE: Executing a long-running zfs program in the background
11:03:24.89 Channel program fully executed and produced output:
11:03:24.89 return: 'should not have reached here'
11:03:24.94
11:03:24.94 ERROR: pkill -P 18751 exited 1
11:03:24.96 kill: 18751: no such process
11:03:24.96 ERROR: kill 18751 exited 1
11:03:25.04 NOTE: 100 snapshots created by ZCP
11:03:25.04 NOTE: Performing test-fail callback (/opt/zfs-tests/callbacks/zfs_dbgmsg)

#4

Updated by Electric Monk 10 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit cdd7a662d2c0914e93e1daa44cebae7003c99777

commit  cdd7a662d2c0914e93e1daa44cebae7003c99777
Author: Alexander Pyhalov <apyhalov@gmail.com>
Date:   2020-02-05T14:34:36.000Z

    12254 python changes have broken the zfs test suite
    Reviewed by: Andy Fiddaman <omnios@citrus-it.co.uk>
    Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

#5

Updated by Andrew Stormont 9 months ago

This seems to have broken Python 2.7.X.

#6

Updated by Yuri Pankov 8 months ago

https://pythonclock.org/

May be it's just about time to forget about 2.x?

#7

Updated by Andrew Stormont 8 months ago

  • Related to Bug #12397: 12254 broke the zfs test suite for older python versions added

Also available in: Atom PDF