Bug #10052
closed
"dladm show-ether" should pick one kstat snapshot and stick with it
100%
Description
When dladm show-ether is invoked, especially on more recent systems, there is a long pause before information about each NIC is emitted. As it turns out, the command opens and closes the kstat device ~56 times per NIC.
System with one NIC:
$ pfexec time dtrace -c 'dladm show-ether' -n 'syscall::open:entry /pid == $target/ { self->x = arg0; } syscall::open:return /arg0 >= 0 & self->x != NULL/ { @[copyinstr(self->x)] = count(); } syscall::open:return /self->x != NULL/ { self->x = NULL; }'
dtrace: description 'syscall::open:entry ' matched 3 probes
LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE
vioif0 current up no 1G-f none
dtrace: pid 1969 has exited
/dev/dld 1
/etc/svc/volatile/dladm/dlmgmt_door 1
/usr/lib/locale//en_US.UTF-8/LC_COLLATE/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_CTYPE/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_MESSAGES/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_MONETARY/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_NUMERIC/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_TIME/LCL_DATA 1
/dev/kstat 56
real 2.8
user 1.8
sys 0.9
System with four NICs:
[root@headnode (lab) ~]# time dtrace -c 'dladm show-ether' -n 'syscall::open:entry /pid == $target/ { self->x = arg0; } syscall::open:return /arg0 >= 0 & self->x != NULL/ { @[copyinstr(self->x)] = count(); } syscall::open:return /self->x != NULL/ { self->x = NULL; }'
dtrace: description 'syscall::open:entry ' matched 3 probes
LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE
vioif0 current up no 1G-f none
vioif1 current up no 1G-f none
vioif2 current up no 1G-f none
vioif3 current up no 1G-f none
dtrace: pid 50415 has exited
/dev/dld 1
/etc/svc/volatile/dladm/dlmgmt_door 1
/usr/lib/locale//en_US.UTF-8/LC_COLLATE/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_CTYPE/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_MESSAGES/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_MONETARY/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_NUMERIC/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_TIME/LCL_DATA 1
/dev/kstat 224
real 0m4.714s
user 0m1.086s
sys 0m3.572s
There are several routines in libdladm which kstat_open(), read a single kstat, then kstat_close(); e.g., dladm_get_single_mac_stat(). We should lazily open the kstat handle when required and cache it on the dladm_handle_t, rather than open and close it for each statistic.
Updated by Carlos Neira over 4 years ago
Hi,
This patch avoids repeated calls to kstat_open() in dladm by retaining a kstat_ctl_t pointer in dladm_handle_t, which is then used by kstat consumers.
The libkstat handle is opened when the first caller needs it. This allows us the library to share the kstat handle.
webrev : http://cr.illumos.org/~webrev/cneira/10052/
source files modified for this change.
:100644 100644 757fcb5149 edfc844a94 M usr/src/cmd/flowstat/flowstat.c
:100644 100644 b8e744f4f9 81d3fb2990 M usr/src/lib/libdladm/common/libdladm.c
:100644 100644 c2fceb25ab 36092f89a4 M usr/src/lib/libdladm/common/libdladm.h
:100644 100644 13169285e3 20db1cb1d7 M usr/src/lib/libdladm/common/libdladm_impl.h
:100644 100644 a4f9307bd7 40de09402b M usr/src/lib/libdladm/common/libdlstat.c
:100644 100644 c799723a91 deb9279f4a M usr/src/lib/libdladm/common/libdlstat.h
After changes:
pfexec time dtrace -c 'dladm show-ether' -n 'syscall::open:entry /pid == $target/ { self->x = arg0; } syscall::open:return /arg0 >= 0 & self->x != NULL/ { @[copyinstr(self->x)] = count(); } syscall::open:return /self->x != NULL/ { self->x = NULL; }'
dtrace: description 'syscall::open:entry ' matched 3 probes
LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE
e1000g0 current up yes 1G-f bi
dtrace: pid 109209 has exited
/dev/dld 1
/dev/kstat 1
/etc/svc/volatile/dladm/dlmgmt_door 1
/usr/lib/locale//en_US.UTF-8/LC_COLLATE/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_CTYPE/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_MESSAGES/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_MONETARY/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_NUMERIC/LCL_DATA 1
/usr/lib/locale//en_US.UTF-8/LC_TIME/LCL_DATA 1
real 0.4
user 0.6
sys 0.7
cneira@Trixie:…os/usr/src/lib/libdladm/common$ dlstat -i 10
LINK IPKTS RBYTES OPKTS OBYTES
e1000g0 108.83K 10.32M 157.52K 18.15M
vnic1 1.56K 84.66K 0 0
vnic0 1.56K 84.66K 0 0
vnic2 1.56K 84.66K 0 0
net6 1.56K 84.66K 0 0
e1000g0 49 3.19K 90 10.25K
vnic1 0 0 0 0
vnic0 0 0 0 0
vnic2 0 0 0 0
net6 0 0 0 0
e1000g0 47 3.16K 85 9.81K
vnic1 0 0 0 0
vnic0 0 0 0 0
vnic2 0 0 0 0
net6 0 0 0 0
e1000g0 51 3.31K 92 9.99K
vnic1 2 84 0 0
vnic0 2 84 0 0
vnic2 2 84 0 0
net6 2 84 0 0
e1000g0 51 3.31K 91 10.23K
vnic1 0 0 0 0
vnic0 0 0 0 0
vnic2 0 0 0 0
net6 0 0 0 0
neira@Trixie:…os/usr/src/lib/libdladm/common$ pfexec flowadm add-flow -l e1000g0 -a transport=tcp,local_port=22 sshd
cneira@Trixie:~$ flowstat -i 1
FLOW IPKTS RBYTES IERRS OPKTS OBYTES OERRS
smtp0 0 0 0 0 0 0
sshd 15.44K 1.11M 0 22.44K 2.57M 0
smtp0 0 0 0 0 0 0
sshd 8 564 0 11 1.28K 0
smtp0 0 0 0 0 0 0
sshd 5 300 0 8 960 0
smtp0 0 0 0 0 0 0
sshd 5 300 0 9 998 0
smtp0 0 0 0 0 0 0
sshd 8 564 0 10 1.12K 0
smtp0 0 0 0 0 0 0
sshd 4 240 0 7 906 0
smtp0 0 0 0 0 0 0
sshd 5 300 0 8 1.01K 0
smtp0 0 0 0 0 0 0
sshd 7 504 0 9 1.03K 0
smtp0 0 0 0 0 0 0
sshd 5 300 0 10 1.18K 0
smtp0 0 0 0 0 0 0
sshd 5 300 0 8 944 0
cneira@Trixie:~$
Joshua Clulow wrote:
When
dladm show-etheris invoked, especially on more recent systems, there is a long pause before information about each NIC is emitted. As it turns out, the command opens and closes thekstatdevice ~56 times per NIC.System with one NIC:
[...]
System with four NICs:
[...]
There are several routines in
libdladmwhichkstat_open(), read a single kstat, thenkstat_close(); e.g.,dladm_get_single_mac_stat(). We should lazily open the kstat handle when required and cache it on thedladm_handle_t, rather than open and close it for each statistic.
Updated by Electric Monk over 4 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
git commit 399dcf08ed1dc7e6abf8d7e5dc9447e11047c9f2
commit 399dcf08ed1dc7e6abf8d7e5dc9447e11047c9f2
Author: carlos antonio neira bustos <cneirabustos@gmail.com>
Date: 2019-02-11T08:12:27.000Z
10052 "dladm show-ether" should pick one kstat snapshot and stick with it
Reviewed by: Rob Johnston <rob.johnston@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Gergő Mihály Doma <domag02@gmail.com>
Reviewed by: Andy Fiddaman <andy@omniosce.org>
Approved by: Joshua M. Clulow <josh@sysmgr.org>