Project

General

Profile

Actions

Feature #12579

closed

parallelize crypto-test build

Added by Patrick Mooney about 2 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Low
Category:
tests
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

The crypto tests take a long time to build, thanks to a lack of parallelization in the makefile. This can be seen by doing something like ptime dmake install in usr/src/test/crypto-tests/tests/digest and seeing user time almost match wall time:

real     1:49.327840621
user     2:04.840790371
sys         5.600473979

It would be nice to get those components building in parallel so on higher core-count machines, it doesn't slow the build unecessarily.


Related issues

Related to illumos gate - Feature #12595: restructure digest crypto-testsClosedPatrick Mooney

Actions
Actions #1

Updated by Patrick Mooney about 2 years ago

Original timing for dmake install in usr/src/test/crypto-tests/tests:

real     1:56.047726331
user     2:13.930877813
sys         8.654525907

With added @.PARALLEL@s:

real       37.450739254
user     2:16.227194476
sys         9.716924074

Actions #2

Updated by Patrick Mooney about 2 years ago

From mail_msg on a full vanilla build:

==== Total build time ====

real    0:33:27

And with the patch:
==== Total build time ====

real    0:32:17

A wsdiff of the two proto areas yield only the typical DOF differences.

Actions #3

Updated by Electric Monk about 2 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit e1086107dcddb40ce490ac7c851716c9e1d7796d

commit  e1086107dcddb40ce490ac7c851716c9e1d7796d
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2020-04-20T15:44:12.000Z

    12579 parallelize crypto-test build
    Reviewed by: Jason King <jason.king@joyent.com>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Robert Mustacchi <rm@fingolfin.org>

Actions #4

Updated by Joshua M. Clulow about 2 years ago

  • Status changed from Closed to In Progress
Actions #5

Updated by Electric Monk about 2 years ago

git commit 865498e43471404cd766389d4b8e045ed6ef3be1

commit  865498e43471404cd766389d4b8e045ed6ef3be1
Author: Joshua M. Clulow <josh@sysmgr.org>
Date:   2020-04-22T04:25:40.000Z

    backout: 12579 parallelize crypto-test build (needs more work)

Actions #6

Updated by Joshua M. Clulow about 2 years ago

This was backed out because it appears to have caused at least one build failure. From Dan:

==== Build errors (DEBUG) ====

The following command caused the error:
The following command caused the error:

==== Build warnings (DEBUG) ====

dmake: Warning: Command failed for target `md5_64_pkcs'
dmake: Warning: Target `install' not remade because of errors
dmake: Warning: Command failed for target `md5'
dmake: Warning: Target `install' not remade because of errors
dmake: Warning: Target `install' not remade because of errors
dmake: Warning: Command failed for target `tests'
dmake: Warning: Target `install' not remade because of errors
dmake: Warning: Command failed for target `crypto-tests'
dmake: Warning: Target `install' not remade because of errors
dmake: Warning: Target `install' not remade because of errors

See also, the advocates thread: https://illumos.topicbox.com/groups/advocates/T0dff3a5173e0cd6a-M9c0ce3b07889c6785de5ec68

Actions #7

Updated by Patrick Mooney about 2 years ago

Actions #9

Updated by Patrick Mooney about 2 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 100 to 0
Actions

Also available in: Atom PDF