Project

General

Profile

Feature #13248

parallelise the quest for elves

Added by Andy Fiddaman about 1 month ago. Updated 10 days ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
tools - gate/build tools
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

The ELF verification and runtime checks from nightly are a serial part of the gate build which take a while to run.

The find_elf and check_rtime utilities could be improved to use multiple threads where supported by the local system perl (and fall back to the current serial behaviour otherwise)

This issue concerns find_elf, a separate issue will be raised for check_rtime

#1

Updated by Electric Monk about 1 month ago

  • Gerrit CR set to 1011
#2

Updated by Andy Fiddaman about 1 month ago

  • Description updated (diff)
  • Gerrit CR deleted (1011)
#3

Updated by Andy Fiddaman about 1 month ago

  • Gerrit CR set to 1011
#4

Updated by Andy Fiddaman about 1 month ago

Some benchmarks of the old and new variant on OmniOS r151036 with perl 5.32 (which has threads support) - DMAKE_JOBS was set to 8 and prstat -L showed 9 threads present and correct.

Old:

% hyperfine -w 1 -r 5 'find_elf /usr'
  Time (mean):     543.760 s +/- 13.640 s    [User: 168.792 s, System: 367.286 s]
  Range (min | max):   528.942 s | 557.340 s    5 runs

New:

  Time (mean):     274.048 s +/-  1.902 s    [User: 281.701 s, System: 779.570 s]
  Range (min | max):   272.608 s | 277.256 s    5 runs

I also diffed the output of the old and new find_elf - it was unchanged.

#5

Updated by Electric Monk 10 days ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit ef1b64136eb11e65f9eb27b6083536fe1b69c707

commit  ef1b64136eb11e65f9eb27b6083536fe1b69c707
Author: Dominik Hassler <hadfl@omnios.org>
Date:   2020-11-19T21:56:56.000Z

    13248 parallelise the quest for elves
    Reviewed by: Andy Fiddaman <andy@omnios.org>
    Reviewed by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF