Project

General

Profile

Bug #11547

Want connstat(1M) command to display per-connection TCP statistics

Added by Cody Mello 7 days ago. Updated about 1 hour ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
networking
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:

Description

Delphix created a connstat(1M) command, which allows fetching statistics about individual TCP connections. The relevant Delphix commits are:

- DLPX-37540/DLPX-37544 connstat command to display per-connection TCP statistics
- DLPX-43064 include high-resolution round-trip times in connstat
- DLPX-45049 include unsent bytes in connstat output
- DLPX-44739 connstat rtt field should be 0 for connections that have not sent data
- DLPX-45697 Adding Avg. RTT to connstat
- DLPX-39339/DLPX-45914/DLPX-45941 (fixes to connstat plus a manual page)

This work was initially done by Sebastien Roy and Ahmed G at Delphix, with some further changes made by Cody Mello at Joyent as part of OS-7327.

History

#1

Updated by Cody Mello 3 days ago

Filtering on the STATE field works:

cpm@utu ~ % connstat -F state=listen
LADDR                     LPORT RADDR                     RPORT STATE
0.0.0.0                   22    0.0.0.0                   0     LISTEN
0.0.0.0                   111   0.0.0.0                   0     LISTEN
0.0.0.0                   111   0.0.0.0                   0     LISTEN
0.0.0.0                   44393 0.0.0.0                   0     LISTEN
0.0.0.0                   52920 0.0.0.0                   0     LISTEN
::                        22    ::                        0     LISTEN
::                        111   ::                        0     LISTEN
::                        52920 ::                        0     LISTEN
::1                       8080  ::                        0     LISTEN

Show only the IPv4 connections:

cpm@utu ~ % connstat -F state=listen -4
LADDR                     LPORT RADDR                     RPORT STATE
0.0.0.0                   22    0.0.0.0                   0     LISTEN
0.0.0.0                   111   0.0.0.0                   0     LISTEN
0.0.0.0                   111   0.0.0.0                   0     LISTEN
0.0.0.0                   44393 0.0.0.0                   0     LISTEN
0.0.0.0                   52920 0.0.0.0                   0     LISTEN

And only the IPv6 connections:

cpm@utu ~ % connstat -F state=listen -6
LADDR                     LPORT RADDR                     RPORT STATE
::                        22    ::                        0     LISTEN
::                        111   ::                        0     LISTEN
::                        52920 ::                        0     LISTEN
::1                       8080  ::                        0     LISTEN

Print stats three times in a row separated by two seconds with a timestamp:

cpm@utu ~ % connstat -F state=listen -6 -c 3 -i 2 -T d
August 15, 2019 at 10:16:25 PM PDT
LADDR                     LPORT RADDR                     RPORT STATE
::                        22    ::                        0     LISTEN
::                        111   ::                        0     LISTEN
::                        52920 ::                        0     LISTEN
::1                       8080  ::                        0     LISTEN
August 15, 2019 at 10:16:27 PM PDT
LADDR                     LPORT RADDR                     RPORT STATE
::                        22    ::                        0     LISTEN
::                        111   ::                        0     LISTEN
::                        52920 ::                        0     LISTEN
::1                       8080  ::                        0     LISTEN
August 15, 2019 at 10:16:29 PM PDT
LADDR                     LPORT RADDR                     RPORT STATE
::                        22    ::                        0     LISTEN
::                        111   ::                        0     LISTEN
::                        52920 ::                        0     LISTEN
::1                       8080  ::                        0     LISTEN

Same thing, but this time parsable and showing the connection's MSS:

cpm@utu ~ % connstat -F state=listen -6 -c 3 -i 2 -T d -P -o laddr,lport,mss
= August 15, 2019 at 10:17:55 PM PDT
::,22,1220
::,111,1220
::,52920,1220
::1,8080,1220
= August 15, 2019 at 10:17:57 PM PDT
::,22,1220
::,111,1220
::,52920,1220
::1,8080,1220
= August 15, 2019 at 10:17:59 PM PDT
::,22,1220
::,111,1220
::,52920,1220
::1,8080,1220

Adding -L removes the loopback addresses:

cpm@utu ~ % connstat -L
LADDR                     LPORT RADDR                     RPORT STATE
0.0.0.0                   22    0.0.0.0                   0     LISTEN
0.0.0.0                   111   0.0.0.0                   0     LISTEN
0.0.0.0                   0     0.0.0.0                   0     IDLE
0.0.0.0                   111   0.0.0.0                   0     LISTEN
0.0.0.0                   0     0.0.0.0                   0     IDLE
0.0.0.0                   44393 0.0.0.0                   0     LISTEN
0.0.0.0                   52920 0.0.0.0                   0     LISTEN
10.88.88.201              22    10.88.88.2                57016 ESTABLISHED
::                        22    ::                        0     LISTEN
::                        111   ::                        0     LISTEN
::                        0     ::                        0     IDLE
::                        52920 ::                        0     LISTEN

Using -e shows only established:

cpm@utu ~ % connstat -L -e
LADDR                     LPORT RADDR                     RPORT STATE
10.88.88.201              22    10.88.88.2                57016 ESTABLISHED

Trying to combine that with a filter on STATE produces an error:

cpm@utu ~ % connstat -L -e -F state=listen
Ambiguous filter provided. The "state" field appears more than once.
usage: connstat [-eLP] [-4|-6] [-T d|u] [-F <filter>]
               [-i <interval> [-c <count>]] [-o <field>[,...]]
<snip further usage...>

This change adds new fields to struct tcpConnEntryInfo_s, which is consumed by netstat, so I ran the netstat command to show that connection statistics are still printed correctly:

cpm@utu ~ % netstat -nv -P tcp          

TCP: IPv4
Local/Remote Address Swind  Snext     Suna   Rwind  Rnext     Rack    Rto   Mss     State
-------------------- ----- -------- -------- ----- -------- -------- ----- ----- -----------
10.88.88.201.22     
10.88.88.2.57016     182272 00000000 00000000 128872 00000000 00000000   400  1448 ESTABLISHED

TCP: IPv6
Local/Remote Address              Swind  Snext     Suna   Rwind  Rnext     Rack    Rto   Mss    State      If
--------------------------------- ----- -------- -------- ----- -------- -------- ----- ----- ----------- -----
::1.35737                        
::1.8080                          65440 00000000 00000000 139264 00000000 00000000  1125  8180 ESTABLISHED 
::1.8080                         
::1.35737                         139060 00000000 00000000 130880 00000000 00000000  1125  8180 ESTABLISHED 

Active UNIX domain sockets
Address  Type          Vnode     Conn  Local Addr      Remote Addr
ffffff03fd16f3e0 stream-ord 0000000 0000000                               
ffffff04006bf7c8 stream-ord 0000000 0000000                               
ffffff04006bfba0 stream-ord 0000000 0000000                               
ffffff0404ea8bb0 stream-ord ffffff0404eb0400 0000000 /var/run/.inetd.uds                
ffffff04006b9020 dgram      ffffff0406273300 0000000 /var/run/in.ndpd_mib                
ffffff04006b93f8 stream-ord ffffff03f433e800 0000000 /var/run/in.ndpd_ipadm
#2

Updated by Electric Monk about 1 hour ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit a2f04351e04971ab0879872d264d6038c156b860

commit  a2f04351e04971ab0879872d264d6038c156b860
Author: Sebastien Roy <seb@delphix.com>
Date:   2019-08-19T22:32:46.000Z

    11547 Want connstat(1M) command to display per-connection TCP statistics
    Portions contributed by: Cody Peter Mello <cody.mello@joyent.com>
    Portions contributed by: Ahmed G <ahmedg@delphix.com>
    Reviewed by: Jason King <jason.king@joyent.com>
    Reviewed by: Robert Mustacchi <rm@joyent.com>
    Reviewed by: Dan McDonald <danmcd@joyent.com>
    Approved by: Richard Lowe <richlowe@richlowe.net>

Also available in: Atom PDF