illumos gate

I have found this commit by Bill Pijewski which updates illumos to expose the per-zone tcp kstats into the global zone, which looks like a basis for perhaps doing the same in other modules:

4303 want local zone TCP kstats in the global zone
https://github.com/illumos/illumos-gate/commit/4a0bd0711d2aa8c9386d9c275964fd49a79f7ea5

Hopefully this is helpful.

Following the model of the patch in 4303, I've prepared: https://gist.github.com/alaslums/807185d21a964a3824e0abf74c2ea810

This fixes up the following modules:

icmp
ip
ip6
sctp
udp

ilb and ipf can have multiple instances per netstack, so the indexing by stackid falls down. ipnet doesn't use kstat_create_netstack so I've left it alone.

I don't currently have a facility for building illumos-gate but will see about setting one up so I can test the changes.

I've now done this with ipfilter:

https://gist.github.com/alaslums/f609af779f723607e0e20388437964ff

To avoid repeated calls to net_getnetstackidbynetid, I've switched from net_kstat_create to kstat_create_netstack (which is used in most places anyway). I'm not sure if this is the right approach as net_kstat_create is a documented interface whereas kstat_create_netstack isn't.

I'm also not sure about the "error handling" approach of wrapping things in a big if block, but this approach was already used in this function (and presumably you'll never hit the case where a stackid can't be found, and if we do then this kstat doesn't get initialised which isn't the end of the world).

Hi Alasdair,

I took your changes and tested them in Omniosce they are working for me. I just submit them for review.

https://illumos.org/rb/r/1474/

With your changes stats are exposed to GZ :

cneira@Trixie:/build/illumos-omnios$ zoneadm list -icv
  ID NAME             STATUS     PATH                           BRAND    IP
   0 global           running    /                              ipkg     shared
   1 t0               running    /zcage/vms/t0                  lx       excl
   - 9e19b8ed-ad9c-4284-a948-3cf803aaa3e0 configured /zcage/vms/9e19b8ed-ad9c-4284-a948-3cf803aaa3e0 lx       excl
   - 1fd17810-1517-47ad-8389-3557b74bdb11 configured /zcage/vms/1fd17810-1517-47ad-8389-3557b74bdb11 lx       excl
   - lxvm0            configured /zcage/vms/lxvm0               lx       excl
   - lxvm             configured /zcage/vms/lxvm                lx       excl
   - lxvm2            configured /zcage/vms/lxvm2               lx       excl
   - l1               configured /zcage/vms/l1                  lx       excl
cneira@Trixie:/build/illumos-omnios$ kstat -p ip:*:icmp:outMsgs
ip:0:icmp:outMsgs       0
ip:1:icmp:outMsgs       0
cneira@Trixie:/build/illumos-omnios$ kstat -p udp:*:udp:inDatagrams
udp:0:udp:inDatagrams   33
udp:1:udp:inDatagrams   0
cneira@Trixie:/build/illumos-omnios$ kstat -p tcp:*:tcp:inAckBytes
tcp:0:tcp:inAckBytes    321216
tcp:1:tcp:inAckBytes    0
omnios$ kstat -p sctp:*:sctp:sctpRtoMax
sctp:0:sctp:sctpRtoMax  60000
sctp:1:sctp:sctpRtoMax  60000

Bests

Alasdair Lumsden wrote:

I've now done this with ipfilter:

https://gist.github.com/alaslums/f609af779f723607e0e20388437964ff

To avoid repeated calls to net_getnetstackidbynetid, I've switched from net_kstat_create to kstat_create_netstack (which is used in most places anyway). I'm not sure if this is the right approach as net_kstat_create is a documented interface whereas kstat_create_netstack isn't.

I'm also not sure about the "error handling" approach of wrapping things in a big if block, but this approach was already used in this function (and presumably you'll never hit the case where a stackid can't be found, and if we do then this kstat doesn't get initialised which isn't the end of the world).