Project

General

Profile

Bug #11394

iconv_modules build could be much more parallel

Added by Rich Lowe 3 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
tools - gate/build tools
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:

Description

usr/src/lib/iconv_modules builds very serially for no real reason, at least in my builds it often takes longer than all the rest of usr/src/lib together, and is delaying the move into building usr/src/cmd

Trivial parallelism improves this radically (almost 2x), and buys a minute or two.

History

#1

Updated by Electric Monk 3 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 270832fa1e63c01a4a8ecb6dc23eb148a4d54831

commit  270832fa1e63c01a4a8ecb6dc23eb148a4d54831
Author: Richard Lowe <richlowe@richlowe.net>
Date:   2019-07-16T21:54:57.000Z

    11394 iconv_modules build could be much more parallel
    Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
    Approved by: Robert Mustacchi <rm@joyent.com>

#2

Updated by Rich Lowe 3 months ago

There's a narrow race Mike Gerdts hit and debugged in this, it runs thuswise:

utf-8/Makefile.com (and probably ja/Makefile.com) define an install rule that rather than being dependency based, loops over files to do the installation.

This rule causes the build of what are conceptually common files, and because it's in Makefile.com causes them to be built per platform unnecessarily. This causes them to install from both platform directories, now possibly simultaneously.

Because this loop does rm/install/chmod, it leaves a window where chmod may have the file removed underneath it by the install in the other directory.

We need to re-write these targets properly, and to have them not run unnecessarily in each platform

#3

Updated by Electric Monk 3 months ago

git commit 01d6bbace7ed9d92acc6fd5dfcb2b18c2dd18e85

commit  01d6bbace7ed9d92acc6fd5dfcb2b18c2dd18e85
Author: Robert Mustacchi <rm@joyent.com>
Date:   2019-07-17T22:09:37.000Z

    backout: 11394 iconv_modules build could be much more parallel (breaks build)

#4

Updated by Rich Lowe 3 months ago

Oh, and of course you can't rewrite these rules properly, because make(1S) provides no way to escape a '%'. The best you can do is trust it to only match itself

#5

Updated by Electric Monk about 2 months ago

git commit dfe6d73f5cf5922a7dfdafeac8df9775afb2d992

commit  dfe6d73f5cf5922a7dfdafeac8df9775afb2d992
Author: Richard Lowe <richlowe@richlowe.net>
Date:   2019-08-22T23:22:43.000Z

    11394 iconv_modules build could be much more parallel
    Reviewed by: Andy Fiddaman <andy@omniosce.org>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Robert Mustacchi <rm@joyent.com>

Also available in: Atom PDF