Bug #5468
closed
Missing dependencies in lib/Makefile
100%
Description
I'm doing this build on a T2000, which has 32 virtual CPUs. I've seen dozens of dmake processes running simultaneously. One result of this parallelism is that some libraries are linked with other libraries on the build host because these other libraries have not been built and installed yet in the proto area. Often you can't tell that this has happened when the build host is also an illumos system. I was building on a Solaris 11.2 system, making it obvious because of a new symbol.
I've noticed that many libraries are missing dependancies on other libraries in lib/Makefile . Adding some of them was an improvement, but I never knew that I got all of them. Something that would limit library search to specified paths would be a better way of finding what's missing.
Files
Related issues
Updated by Gary Mills over 8 years ago
I'm attaching a patch file that shows the dependancies I've added to the Makefile. I only fixed some targets, though. Likely I've missed many more.
Updated by Rich Lowe over 8 years ago
At one point I had patches that made this better, and were likely very similar to yours (I didn't look).
All I did was grep the Makefiles for '-l' and state the dependencies so discovered.
I'd say it's totally worth fixing these piecemeal, if you wanted, as long as the bug made it clear there were still problems remaining
Updated by Andrew Stormont over 8 years ago
A few that you've missed:
- libdladm depends on libpool
- librestart depends on libpool
- libdtrace depends on libelf, libmapmalloc
- libmapmalloc depends on libc
- libmp depends on libc
- libkvm depends on libc, libelf
- libtnfctl depends on libc, libelf
- libelfsign depends on libelf, libmd
Also in your patch you have two copyright notices.
Updated by Gary Mills over 8 years ago
I believe that it's not necessary to include libc in dependancy lists of this file; there is an explicit .WAIT target to ensure that libc is built first.
The first Copyright notice is historical, from a previous change I made. Should I have deleted it?
Updated by Andrew Stormont over 8 years ago
Without the libc dependency it seems to be pulling in the host version of these libraries. I'm also building on a machine with 32 logical cores (16 core Xeon with hyper threading).
Generally we update the copyright notices instead of adding a new one for each year - though there are times when you see a copyright notice span multiple years - e.g "Copyright 2014-2014 Andrew Stormont". I pointed it out because it looked like a mistake.
Updated by Andrew Stormont over 8 years ago
- Priority changed from Normal to High
The problem is actually worse than originally thought. Any time a make target uses $(LINK.c) but does not pass $(LDLIBS) we are linking against libraries outside the proto area. I think it's about time illumos started passing -zfatal-warnings and -zassert-deflibs in the default LDFLAGS to catch these and other linker errors.
Updated by Andrew Stormont over 8 years ago
- Subject changed from Missing dependancies in lib/Makefile to Missing dependencies in lib/Makefile
Updated by Andrew Stormont over 8 years ago
Here is a webrev which fixes all instances of the build linking against libraries outside of the proto area: http://cr.illumos.org/~webrev/andy_js/5468/
I have not done a clean build so I don't know if there are any issues, but I think it's a good first step. Please review it when you get a chance.
Updated by Gary Mills over 8 years ago
After a clean build on x86, I tried applying your patch and doing another build. There were no build errors, but I did get a bunch of ELF errors. I've attached elf.err for these.
I also ran wsdiff, comparing the prototype area before and after. Here's how it looked:
<mills@ati:941>$ env PATH=/opt/onbld/bin:$PATH wsdiff -r /tmp/1.res proto-bas> usr/share/doc/ksh/shell_styleguide.html usr/sbin/isns usr/demo/link_audit/Makefile usr/lib/libc/libc_hwcap3.so.1 usr/lib/libc/libc_hwcap2.so.1 usr/lib/libc/libc_hwcap1.so.1 usr/lib/netsvc/yp/udpublickey usr/lib/netsvc/yp/stdethers usr/lib/nfs/mountd usr/lib/picl/plugins/libpicldevtree.so.1 usr/lib/amd64/libzpool.so.1 usr/lib/dtrace/64/libdtrace_forceload.so usr/lib/dtrace/libdtrace_forceload.so usr/lib/libzpool.so.1 etc/motd lib/amd64/libc.so.1 lib/libc.so.1 <mills@ati:943>$ grep ^NOTE /tmp/1.res NOTE: HTML difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ASCII difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ELF .text difference detected. NOTE: ELF .text difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ELF .text difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ELF .text difference detected. NOTE: ELF .text difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ASCII difference detected. NOTE: ELF .SUNW_dof difference detected. NOTE: ELF .SUNW_dof difference detected.
The files that had the .text differences were these:
usr/lib/netsvc/yp/udpublickey usr/lib/netsvc/yp/stdethers usr/lib/picl/plugins/libpicldevtree.so.1 usr/lib/dtrace/64/libdtrace_forceload.so usr/lib/dtrace/libdtrace_forceload.so
Updated by Andrew Stormont over 8 years ago
The .SUNW_dof stuff might be noise if Rich is right in #729.
Updated by Gary Mills over 8 years ago
Yes, only the .text differences are significant here. I suspect they will disappear when the ELF errors are resolved.
Updated by Andrew Stormont over 8 years ago
- Related to Bug #5837: libdtrace audit library build missing LDLIBS added
Updated by Yuri Pankov almost 8 years ago
- Status changed from Feedback to In Progress
- Assignee set to Yuri Pankov
- % Done changed from 20 to 50
- Difficulty changed from Medium to Bite-size
- Tags deleted (
needs-triage)
I have what looks to be complete fix for this. Andrew's changes to add missing LDLIBS should (and will) be handled in separate issue.
Updated by Yuri Pankov almost 8 years ago
- Status changed from In Progress to Pending RTI
- % Done changed from 50 to 90
Updated by Electric Monk almost 8 years ago
- Status changed from Pending RTI to Closed
- % Done changed from 90 to 100
git commit ba9ca9119d43e9c8e21e14f2129dce27d2dc0cdc
commit ba9ca9119d43e9c8e21e14f2129dce27d2dc0cdc
Author: Yuri Pankov <yuri.pankov@nexenta.com>
Date: 2015-12-05T14:50:53.000Z
5468 Missing dependencies in lib/Makefile
Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Reviewed by: Gary Mills <gary_mills@fastmail.fm>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Hans Rosenfeld <hans.rosenfeld@nexenta.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Updated by Electric Monk almost 8 years ago
git commit b5254d6ec12a0443b3a431a7e656f38e4ac98b2e
commit b5254d6ec12a0443b3a431a7e656f38e4ac98b2e
Author: Yuri Pankov <yuri.pankov@nexenta.com>
Date: 2015-12-14T22:04:05.000Z
5468 Missing dependencies in lib/Makefile (fix scsi)