Bug #12819
pkgmogrify should not remove quotes from values containing macros
0%
Description
There has been a long-standing issue with pkgmogrify producing invalid intermediate manifests e.g. for different Python versions.
For macros potentially containing whitespaces such as COMPONENT_SUMMARY or COMPONENT_LICENSE the following error may occur at the publish stage:
pkgmogrify: File /scratch/alarcher/oi-userland/components/python/colorama/build/manifest-i386-colorama-35.p5m line 17: Malformed action at position: 48: whitespace in key: set name=pkg.summary value=Simple cross-platform colored terminal text in Python
The reason lies in the generation of intermediate manifests where only some macros are substituted but not all:
/usr/bin/pkgmogrify -D PYVER=3.5 -D PY3_ABI3_NAMING="#" -D PY3_CYTHON_NAMING="" -D PYV=35 colorama-PYVER.p5m > /scratch/alarcher/oi-userland/components/python/colorama/build/manifest-i386-colorama-35.p5m
Comparing the original PYVER manifest and the versioned one:
--- colorama-PYVER.p5m 2020-06-04 12:53:28.363511839 +0000 +++ build/manifest-i386-colorama-35.p5m 2020-06-04 16:19:20.261780928 +0000 @@ -10,25 +10,25 @@ # # -# Copyright 2019 Nona Hansel +# Copyright 2019 Nona Hansel # -set name=pkg.fmri value=pkg:/$(COMPONENT_FMRI)-$(PYV)@$(IPS_COMPONENT_VERSION),$(BUILD_VERSION) -set name=pkg.summary value="$(COMPONENT_SUMMARY)" -set name=info.classification value="$(COMPONENT_CLASSIFICATION)" +set name=pkg.fmri value=pkg:/$(COMPONENT_FMRI)-35@$(IPS_COMPONENT_VERSION),$(BUILD_VERSION) +set name=pkg.summary value=$(COMPONENT_SUMMARY) +set name=info.classification value=$(COMPONENT_CLASSIFICATION) set name=info.upstream-url value=$(COMPONENT_PROJECT_URL) set name=info.source-url value=$(COMPONENT_ARCHIVE_URL) set name=org.opensolaris.consolidation value=$(CONSOLIDATION) -license $(COMPONENT_LICENSE_FILE) license="$(COMPONENT_LICENSE)" +license $(COMPONENT_LICENSE_FILE) license=$(COMPONENT_LICENSE)
Issues occur during the substitution since pkg5 automatically formats action values since:
- quotes are systematically removed in
fromstr()
during the construction of the action dictionary, - quotes are automatically added if the action value contains a whitespace character or if it is already enclosed in quotes.
The first assumption causes failure of multiple stage substitution with action values containing whitespace characters while the second adds spurious quotes around an already quoted value.
To avoid any breakage of the existing behaviour and since it involves only intermediate manifest generation, we could add a flag -r
to pkgmogrify to preserve raw action values e.g. keep quotes around macro values that are not substituted at this stage:
- adding an argument to
fromstr()
in_action.c
which instructs to keep the quotes (1), - avoiding to add quotes to macro values already enclosed in quotes (2),
- plumbing the flags and updating the documentation.
The default behaviour is unchanged but a change should be made in oi-userland
to pass -r
for intermediate stages.
narval> diff -uN default.p5m raw.p5m --- default.p5m 2020-06-04 15:43:52.893562776 +0000 +++ raw.p5m 2020-06-04 15:56:09.219708229 +0000 @@ -14,13 +14,13 @@ # set name=pkg.fmri value=pkg:/$(COMPONENT_FMRI)-35@$(IPS_COMPONENT_VERSION),$(BUILD_VERSION) -set name=pkg.summary value=$(COMPONENT_SUMMARY) -set name=info.classification value=$(COMPONENT_CLASSIFICATION) +set name=pkg.summary value="$(COMPONENT_SUMMARY)" +set name=info.classification value="$(COMPONENT_CLASSIFICATION)" set name=info.upstream-url value=$(COMPONENT_PROJECT_URL) set name=info.source-url value=$(COMPONENT_ARCHIVE_URL) set name=org.opensolaris.consolidation value=$(CONSOLIDATION) -license $(COMPONENT_LICENSE_FILE) license=$(COMPONENT_LICENSE) +license $(COMPONENT_LICENSE_FILE) license="$(COMPONENT_LICENSE)" file NOHASH path=usr/lib/python3.5/vendor-packages/colorama-$(COMPONENT_VERSION)-py3.5.egg-info/PKG-INFO file NOHASH path=usr/lib/python3.5/vendor-packages/colorama-$(COMPONENT_VERSION)-py3.5.egg-info/SOURCES.txt
Another preferred solution which is less intrusive would be to re-add quotes by default to values containing macros, based on the presence of '
or "
in the string.
Updated by Aurélien Larcher 7 months ago
- Subject changed from pkgmogrify could conditionally preserve raw action values to pkgmogrify should not remove quotes from values containing macros
- Description updated (diff)