Bug #6697

Aggregated IOPS, bandwidth, and latency kstats for NFS server

Added by Marcel Telka over 3 years ago. Updated about 3 years ago.

nfs - NFS server and client
Start date:
Due date:
% Done:


Estimated time:


As an enhancement to the detailed NFS server kstats (#6090 and #6696) we would like to have the aggregated NFS server kstats broken down as read/write/other and usable to calculate aggregated IOPS, bandwidth, and latency.

Related issues

Related to illumos gate - Bug #6747: xdr_READDIR4res() bypass the XDR mblk APIClosed2016-03-13

Related to illumos gate - Bug #6696: Per-client NFS server IOPS, bandwidth, and latency kstatsRejected2016-02-29




Updated by Marcel Telka over 3 years ago

Aggregated NFS server stats


There are four types of aggregated kstats implemented:

Type Module Instance Name Class
per NFS server (global) stats nfssrv 0 aggregates aggregates
per-share stats SHARE_ID share_aggregates
per-client stats nfssrv_client CLIENT_ID 0 client_aggregates
per-client/per-share stats SHARE_ID aggregates

Every aggregated stat is a named kstat with these entries:

Name Comment
read.rnum Number of received read operations
read.tnum Number of sent replies to read operations
read.rx Number of bytes received in the read operations
read.tx Number of bytes sent in the replies to read operations
read.time Accumulated processing time for reads
write.rnum Number of received write operations
write.tnum Number of sent replies to write operations
write.rx Number of bytes received in the write operations
write.tx Number of bytes sent in the replies to write operations
write.time Accumulated processing time for writes
other.rnum Number of received other operations
other.tnum Number of sent replies to other operations
other.rx Number of bytes received in the other operations
other.tx Number of bytes sent in the replies to other operations
other.time Accumulated processing time for other operations
  1. The 'other' operations are neither reads (the NFS READ operations), nor writes (the NFS WRITE operations).
  2. The aggregates are aggregated across these protocols: NFS_ACLv2, NFS_ACLv3, NFSv2, NFSv3, and NFSv4.0.

In addition to the aggregated kstats there are generic share and client helper kstats (implemented in #6090 and #6696) to publish shared paths and clients' addresses:

Type Module Instance Name Class
generic per-share stats nfssrv SHARE_ID share misc
generic per-client stats nfssrv CLIENT_ID client misc

Usage example

List of available per-share stats (paths):

# kstat -p -c misc nfssrv::share:path
nfssrv:1:share:path     /
nfssrv:3:share:path     /
nfssrv:4:share:path     /export
nfssrv:5:share:path     /export/home
nfssrv:6:share:path     /export/home/admin
nfssrv:7:share:path     /tmp

Aggregated stats for /tmp (the SHARE_ID is 7):

# kstat -p nfssrv:7:share_aggregates
nfssrv:7:share_aggregates:class aggregates
nfssrv:7:share_aggregates:crtime        10847.260643550
nfssrv:7:share_aggregates:other.rnum    21638
nfssrv:7:share_aggregates:other.rx      552232
nfssrv:7:share_aggregates:other.time    507376083
nfssrv:7:share_aggregates:other.tnum    21638
nfssrv:7:share_aggregates:other.tx      1082664
nfssrv:7:share_aggregates:read.rnum     1
nfssrv:7:share_aggregates:read.rx       32
nfssrv:7:share_aggregates:read.time     56351
nfssrv:7:share_aggregates:read.tnum     1
nfssrv:7:share_aggregates:read.tx       48
nfssrv:7:share_aggregates:snaptime      46271.878494564
nfssrv:7:share_aggregates:write.rnum    769
nfssrv:7:share_aggregates:write.rx      30784
nfssrv:7:share_aggregates:write.time    26270212
nfssrv:7:share_aggregates:write.tnum    769
nfssrv:7:share_aggregates:write.tx      18456

List of available per-client stats (client's addresses):

# kstat -p -c misc nfssrv::client:address

Aggregated stats for client (the CLIENT_ID is 2):

# kstat -p nfssrv_client2:0:client_aggregates
nfssrv_client2:0:client_aggregates:class        aggregates
nfssrv_client2:0:client_aggregates:crtime       46039.841458420
nfssrv_client2:0:client_aggregates:other.rnum   291
nfssrv_client2:0:client_aggregates:other.rx     7312
nfssrv_client2:0:client_aggregates:other.time   1233138813
nfssrv_client2:0:client_aggregates:other.tnum   225
nfssrv_client2:0:client_aggregates:other.tx     13768
nfssrv_client2:0:client_aggregates:read.rnum    1
nfssrv_client2:0:client_aggregates:read.rx      32
nfssrv_client2:0:client_aggregates:read.time    56542
nfssrv_client2:0:client_aggregates:read.tnum    1
nfssrv_client2:0:client_aggregates:read.tx      48
nfssrv_client2:0:client_aggregates:snaptime     46347.563976095
nfssrv_client2:0:client_aggregates:write.rnum   2
nfssrv_client2:0:client_aggregates:write.rx     128
nfssrv_client2:0:client_aggregates:write.time   323578
nfssrv_client2:0:client_aggregates:write.tnum   2
nfssrv_client2:0:client_aggregates:write.tx     48

Aggregated stats for share /tmp from client (the SHARE_ID is 7, the CLIENT_ID is 2):

# kstat -p nfssrv_client2:7:aggregates
nfssrv_client2:7:aggregates:class       aggregates
nfssrv_client2:7:aggregates:crtime      46039.841611529
nfssrv_client2:7:aggregates:other.rnum  121
nfssrv_client2:7:aggregates:other.rx    3484
nfssrv_client2:7:aggregates:other.time  9403887
nfssrv_client2:7:aggregates:other.tnum  121
nfssrv_client2:7:aggregates:other.tx    8076
nfssrv_client2:7:aggregates:read.rnum   1
nfssrv_client2:7:aggregates:read.rx     32
nfssrv_client2:7:aggregates:read.time   56319
nfssrv_client2:7:aggregates:read.tnum   1
nfssrv_client2:7:aggregates:read.tx     48
nfssrv_client2:7:aggregates:snaptime    46271.882492761
nfssrv_client2:7:aggregates:write.rnum  1
nfssrv_client2:7:aggregates:write.rx    64
nfssrv_client2:7:aggregates:write.time  81304
nfssrv_client2:7:aggregates:write.tnum  1
nfssrv_client2:7:aggregates:write.tx    24

Global NFS server aggregated stats:

# kstat -p nfssrv:0:aggregates
nfssrv:0:aggregates:class       aggregates
nfssrv:0:aggregates:crtime      9673.370772820
nfssrv:0:aggregates:other.rnum  129660
nfssrv:0:aggregates:other.rx    2829448
nfssrv:0:aggregates:other.time  26688021206
nfssrv:0:aggregates:other.tnum  91537
nfssrv:0:aggregates:other.tx    4651256
nfssrv:0:aggregates:read.rnum   2
nfssrv:0:aggregates:read.rx     64
nfssrv:0:aggregates:read.time   85902466
nfssrv:0:aggregates:read.tnum   2
nfssrv:0:aggregates:read.tx     72
nfssrv:0:aggregates:snaptime    46994.328855156
nfssrv:0:aggregates:write.rnum  3165
nfssrv:0:aggregates:write.rx    126672
nfssrv:0:aggregates:write.time  347525254
nfssrv:0:aggregates:write.tnum  3165
nfssrv:0:aggregates:write.tx    75960


Updated by Marcel Telka over 3 years ago

  • Related to Bug #6747: xdr_READDIR4res() bypass the XDR mblk API added

Updated by Marcel Telka over 3 years ago

  • Related to Bug #6696: Per-client NFS server IOPS, bandwidth, and latency kstats added

Updated by Marcel Telka about 3 years ago

  • Assignee deleted (Marcel Telka)

Updated by Marcel Telka about 3 years ago

  • Status changed from In Progress to Feedback

Updated by Marcel Telka about 3 years ago

  • Status changed from Feedback to Rejected

Also available in: Atom PDF