illumos gate

This has been tested on OmniOS and OpenIndiana, creating HSFS and UFS archives with both
old and new code and comparing contents, and test booting, and testing CPIO format too.

With this change I'm adding a new property on the `system/boot-archive:default` service to specify the desired archive format; this defaults to cpio. This property is honoured by both the C `bootadm` code, which is responsible for building the HSFS format archives, and by the script that handles UFS and CPIO.
If the property is not present then the default is HSFS if `/usr/bin/mkisofs` is present, and UFS otherwise. If the property is set to an unknown value then UFS will be used.

Additional testing has been performed working on inactive boot environments which have different values for this new SMF property and which have no property set since it is likely that an old BE may not have in-kernel support for booting from CPIO archives; see below.

The man page for `bootadm` has been updated to include information on the new option and also to show the `-f` option for `bootadm update-archive` (force archive creation) which was previously undocumented and the addition of a new `-F` to allow manual specification of the desired format. -F was previously an undocumented flag to flush the boot archive cache directory used for HSFS archive creation, this is now -Q.

On my test system a boot archive contains 1056 files, 92 directories and 216 hardlinks (mostly CPU firmware).

During system boot, 2953 file opens are performed of which 2274 result in ENOENT; the total data read from the archive is 46MiB.

Testing with each of the four archive formats below shows that processing time during boot is in the range 3-9 seconds for all formats, with different boots giving different results. On average archive processing takes around 5 seconds regardless of format.

bloody% svcprop -p config/format boot-archive
cpio

bloody% time pfexec bootadm update-archive -f
updating platform/i86pc/amd64/boot_archive (CPIO)
pfexec bootadm update-archive -f  5.56s user 0.15s system 98% cpu 5.826 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     gzip compressed data - deflate method , original file name
bloody% gzip -dc /platform/i86pc/amd64/boot_archive > /tmp/l
bloody% file /tmp/l
/tmp/l:         ASCII cpio archive - CHR (-c) header
bloody% ls -lh /platform/i86pc/amd64/boot_archive /tmp/l
-rw-r--r--   1 root     root       31.6M Jul  5 15:45 /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 af       other      83.1M Jul  5 15:45 /tmp/l

bloody% svccfg -s boot-archive setprop config/format = ufs
bloody% time pfexec bootadm update-archive -f
updating platform/i86pc/amd64/boot_archive (UFS)
pfexec bootadm update-archive -f  3.86s user 0.92s system 6% cpu 1:15.96 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     English text
bloody% ls -l /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 root     root       45.8M Jul  5 15:48 /platform/i86pc/amd64/boot_archive

bloody% svccfg -s boot-archive setprop config/format = hsfs
bloody% time pfexec bootadm update-archive -f
updating //platform/i86pc/amd64/boot_archive (HSFS)
pfexec bootadm update-archive -f  0.17s user 0.12s system 1% cpu 21.938 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     ISO 9660 filesystem image
bloody% ls -lh /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 root     root       36.4M Jul  5 15:46 /platform/i86pc/amd64/boot_archive

bloody% svccfg -s boot-archive setprop config/format = ufs-nocompress
bloody% time pfexec bootadm update-archive -f
updating platform/i86pc/amd64/boot_archive (UFS-nocompress)
pfexec bootadm update-archive -f  5.32s user 1.54s system 5% cpu 1:56.01 total

bloody% file /platform/i86pc/amd64/boot_archive
/platform/i86pc/amd64/boot_archive:     gzip compressed data - deflate method , original file name
bloody% gzip -dc /platform/i86pc/amd64/boot_archive >| /tmp/l
bloody% file /tmp/l
/tmp/l:         English text
bloody% ls -lh /platform/i86pc/amd64/boot_archive /tmp/l
-rw-r--r--   1 root     root       27.5M Jul  5 15:43 /platform/i86pc/amd64/boot_archive
-rw-r--r--   1 af       other      91.7M Jul  5 15:44 /tmp/l

Also, testing with multiple mounted boot environments with differing properties, including unset (i.e. an old BE)

omni# beadm list | awk '$3 ~ /\// {print $3}' | while read mp; do
echo "$mp: \\c" 
SVCCFG_REPOSITORY=$mp/etc/svc/repository.db svccfg -s boot-archive listprop config/format
done

/: config/format  astring  cpio
/hsfs: config/format  astring  hsfs
/ufs: config/format  astring  ufs
/ufs-nocompress: config/format  astring  ufs-nocompress
/cpio: config/format  astring  cpio
/unset: zsh: exit 1

omni# bootadm -a update_all -f
updating /platform/i86pc/amd64/boot_archive (CPIO)
Creating boot_archive for /hsfs
updating /hsfs//platform/i86pc/amd64/boot_archive (HSFS)
Creating boot_archive for /ufs
updating /ufs/platform/i86pc/amd64/boot_archive (UFS)
Creating boot_archive for /ufs-nocompress
updating /ufs-nocompress/platform/i86pc/amd64/boot_archive (UFS-nocompress)
Creating boot_archive for /cpio
updating /cpio/platform/i86pc/amd64/boot_archive (CPIO)
Creating boot_archive for /unset
updating /unset/platform/i86pc/amd64/boot_archive (UFS)