OpenIndiana Distribution

There has been a long-standing issue with pkgmogrify producing invalid intermediate manifests e.g. for different Python versions.

For macros potentially containing whitespaces such as COMPONENT_SUMMARY or COMPONENT_LICENSE the following error may occur at the publish stage:

pkgmogrify: File /scratch/alarcher/oi-userland/components/python/colorama/build/manifest-i386-colorama-35.p5m line 17: Malformed action at position: 48: whitespace in key:
    set name=pkg.summary value=Simple cross-platform colored terminal text in Python

The reason lies in the generation of intermediate manifests where only some macros are substituted but not all:

/usr/bin/pkgmogrify -D PYVER=3.5 -D PY3_ABI3_NAMING="#" -D PY3_CYTHON_NAMING="" -D PYV=35  colorama-PYVER.p5m > /scratch/alarcher/oi-userland/components/python/colorama/build/manifest-i386-colorama-35.p5m

Comparing the original PYVER manifest and the versioned one:

--- colorama-PYVER.p5m  2020-06-04 12:53:28.363511839 +0000
+++ build/manifest-i386-colorama-35.p5m 2020-06-04 16:19:20.261780928 +0000
@@ -10,25 +10,25 @@
 #

 #
-# Copyright 2019 Nona Hansel
+# Copyright 2019 Nona Hansel
 #

-set name=pkg.fmri value=pkg:/$(COMPONENT_FMRI)-$(PYV)@$(IPS_COMPONENT_VERSION),$(BUILD_VERSION)
-set name=pkg.summary value="$(COMPONENT_SUMMARY)" 
-set name=info.classification value="$(COMPONENT_CLASSIFICATION)" 
+set name=pkg.fmri value=pkg:/$(COMPONENT_FMRI)-35@$(IPS_COMPONENT_VERSION),$(BUILD_VERSION)
+set name=pkg.summary value=$(COMPONENT_SUMMARY)
+set name=info.classification value=$(COMPONENT_CLASSIFICATION)
 set name=info.upstream-url value=$(COMPONENT_PROJECT_URL)
 set name=info.source-url value=$(COMPONENT_ARCHIVE_URL)
 set name=org.opensolaris.consolidation value=$(CONSOLIDATION)

-license $(COMPONENT_LICENSE_FILE) license="$(COMPONENT_LICENSE)" 
+license $(COMPONENT_LICENSE_FILE) license=$(COMPONENT_LICENSE)

Issues occur during the substitution since pkg5 automatically formats action values since:

1. quotes are systematically removed in fromstr() during the construction of the action dictionary,
2. quotes are automatically added if the action value contains a whitespace character or if it is already enclosed in quotes.

The first assumption causes failure of multiple stage substitution with action values containing whitespace characters while the second adds spurious quotes around an already quoted value.

To avoid any breakage of the existing behaviour and since it involves only intermediate manifest generation, we could add a flag -r to pkgmogrify to preserve raw action values e.g. keep quotes around macro values that are not substituted at this stage:

• adding an argument to fromstr() in _action.c which instructs to keep the quotes (1),
• avoiding to add quotes to macro values already enclosed in quotes (2),
• plumbing the flags and updating the documentation.

The default behaviour is unchanged but a change should be made in oi-userland to pass -r for intermediate stages.

narval> diff -uN default.p5m raw.p5m
--- default.p5m 2020-06-04 15:43:52.893562776 +0000
+++ raw.p5m     2020-06-04 15:56:09.219708229 +0000
@@ -14,13 +14,13 @@
 #

 set name=pkg.fmri value=pkg:/$(COMPONENT_FMRI)-35@$(IPS_COMPONENT_VERSION),$(BUILD_VERSION)
-set name=pkg.summary value=$(COMPONENT_SUMMARY)
-set name=info.classification value=$(COMPONENT_CLASSIFICATION)
+set name=pkg.summary value="$(COMPONENT_SUMMARY)" 
+set name=info.classification value="$(COMPONENT_CLASSIFICATION)" 
 set name=info.upstream-url value=$(COMPONENT_PROJECT_URL)
 set name=info.source-url value=$(COMPONENT_ARCHIVE_URL)
 set name=org.opensolaris.consolidation value=$(CONSOLIDATION)

-license $(COMPONENT_LICENSE_FILE) license=$(COMPONENT_LICENSE)
+license $(COMPONENT_LICENSE_FILE) license="$(COMPONENT_LICENSE)" 

 file NOHASH path=usr/lib/python3.5/vendor-packages/colorama-$(COMPONENT_VERSION)-py3.5.egg-info/PKG-INFO
 file NOHASH path=usr/lib/python3.5/vendor-packages/colorama-$(COMPONENT_VERSION)-py3.5.egg-info/SOURCES.txt

Another preferred solution which is less intrusive would be to re-add quotes by default to values containing macros, based on the presence of ' or " in the string.