illumos gate

The following discussion is from Joyent bug OS-2567

As an RFE for the svcadm toolset, we would really like to see `svcadm restart` (all of the operations really) work on a set of instances, rather than just one. As a concrete example - in Manta production, we run 16 instances of the node webapi:

online          3:35:29 svc:/manta/application/registrar:default
online          3:36:30 svc:/manta/application/muskie:muskie-8081
online          3:36:30 svc:/manta/application/muskie:muskie-8082
online          3:36:30 svc:/manta/application/muskie:muskie-8083
online          3:36:30 svc:/manta/application/muskie:muskie-8084
online          3:36:30 svc:/manta/application/muskie:muskie-8085
online          3:36:30 svc:/manta/application/muskie:muskie-8086
online          3:36:30 svc:/manta/application/muskie:muskie-8087
online          3:36:30 svc:/manta/application/muskie:muskie-8089
online          3:36:30 svc:/manta/application/muskie:muskie-8088
online          3:36:30 svc:/manta/application/muskie:muskie-8090
online          3:36:30 svc:/manta/application/muskie:muskie-8092
online          3:36:30 svc:/manta/application/muskie:muskie-8091
online          3:36:30 svc:/manta/application/muskie:muskie-8093
online          3:36:30 svc:/manta/application/muskie:muskie-8094
online          3:36:30 svc:/manta/application/muskie:muskie-8096
online          3:36:32 svc:/manta/application/muskie:muskie-8095
[root@02d02889 (webapi) ~]$

In order to restart that, currently one has to do:

$ for i in {1..9} ; do svcadm restart muskie-808$i ; done
$ for i in {0..6} ; do svcadm restart muskie-809$i ; done

Really, the RFE here is to support:

$ svcadm restart muskie

in an environment like the above – I think it's fine that we say ordering/dependency between them is non-deterministic.

---

That's a pretty big change to the tool's existing semantics. It seems safer to achieve this by adding a wildcard character, e.g.

svcadm restart muskie:*

... would match all instances of muskie.

---

smf has a standard mechanism for matching fmri's, the scf_walk_fmri() function. It is simple to make svcadm handle wildcards, we just add the SCF_WALK_MULTIPLE flag, just like the svcs command does.

However, smf also has builtin shortened fmri handling, so, for example, giving the base name matches multiple instances:

# svcs mdata
STATE          STIME    FMRI
disabled       12:52:58 svc:/smartdc/mdata:fetch
disabled       12:53:01 svc:/smartdc/mdata:execute

If we make svcadm work like svcs already does, then it is hard for me to see how anyone could be depending on the old behavior, since in the case of multiple instances, svcadm was always failing before. However, in a chat with Josh C. he expressed concern about users needing to be protected from themselves. For example, with PostgreSQL, it is delivered with a postgres:pg32 and postgres:pg64 service and it would be weird, or possibly dangerous, to enable both, so if someone ran "svcadm enable postgres" then previously they would have been protected by the failing command but now both would be enabled.

So, we either need to add an option to svcadm which says to enable multiple instance support (lame) or I need to see how hard it will be to change the fmri walking code to disable matching on partial fmri's with a new option. Actually, we can't simply disable that, since it is normal for people to use partial fmri's when they know there is only a single match, so we really need to the new option to error on partial fmri's only when there is more than one, but succeed when there are multiple matches because of wildcards.

---

In the end Jerry was able to replicate he behavior above.