Bug #10524
wsdiff much slower after move from deprecated commands module
Start date:
2019-03-10
Due date:
% Done:
100%
Estimated time:
Difficulty:
Bite-size
Tags:
Gerrit CR:
Description
wsdiff was modified under #9979 so that it could be run under wither Python2 or Python3. Part of this work was to replace code that used the deprecated commands module with the newer subprocess module. This change has inadvertently caused a performance regression.
Here are some timings for running wsdiff against usr/bin with both python2 and python3:
python2 =wsdiff proto/root_i386{,-nd}/usr/bin 467.68s user 238.26s system 281% cpu 4:10.37 total
python2 =wsdiff proto/root_i386{,-nd}/usr/bin 463.40s user 236.09s system 280% cpu 4:09.71 total
python3 =wsdiff proto/root_i386{,-nd}/usr/bin 27.50s user 129.93s system 109% cpu 2:23.35 total
python3 =wsdiff proto/root_i386{,-nd}/usr/bin 27.73s user 129.84s system 110% cpu 2:22.86 total
And with python2 using the version before #9979
wsdiff proto/root_i386{,-nd}/usr/bin 11.84s user 11.74s system 69% cpu 33.974 total
Related issues
Updated by Andy Fiddaman over 1 year ago
- Related to Feature #9979: Support python3 for in-gate tools added
Updated by Andy Fiddaman over 1 year ago
Proposed patch at:
https://illumos.org/rb/r/1554/
Testing notes:
With the patch, python2 wsdiff is much faster (back to previous speed), python3 remains slower.
python2 =wsdiff proto/root_i386{,-nd}/usr/bin 9.22s user 20.69s system 111% cpu 26.866 total
python3 =wsdiff proto/root_i386{,-nd}/usr/bin 31.24s user 101.09s system 110% cpu 2:00.04 total
Updated by Electric Monk over 1 year ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit 2f7dba3e6747cbaaf1deb86e6ca1e2a5c96332ac
commit 2f7dba3e6747cbaaf1deb86e6ca1e2a5c96332ac
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date: 2019-03-14T20:01:47.000Z
10524 wsdiff much slower after move from deprecated commands module
10448 wsdiff explodes on encoding error
10525 wsdiff output is not correct for a binary file
10526 wsdiff tries to spawn 4.8 threads
Reviewed by: Gergő Mihály Doma <domag02@gmail.com>
Reviewed by: Sebastian Wiedenroth <sebastian.wiedenroth@skylime.net>
Approved by: Rich Lowe <richlowe@richlowe.net>