Project

General

Profile

Bug #1333 » 20111118_tcs_ga_after_69hours_dtrace_oddly_high_zfs_activity.txt

other #2 look into unrelated fflush CPU and/or IO issue - Jon Strabala, 2011-11-21 07:34 PM

 
1
#-------------------------------------------------------------------------
2
# runing Dtrace scripts to examine the 'fflush' activity and the 'zpool-rpool'
3
# activity seems to indicate something like a linux issue that was discussed 
4
# at 'https://github.com/zfsonlinux/zfs/issues/210' we have no issues wiht
5
# bug #1333 but I am concerned at this activity on a 100% idle system.  Which
6
# only has a single ssh login to check activity.
7
#
8
# Attached below are th following scripts and the outputs of said scripts almost
9
# all run for 60 seconds (unless otherwise noted):
10
#
11
#    psio10 -x 60
12
#    fsflush.d
13
#    fspaging.d
14
#    bitesize.d
15
#    hotspot.d
16
#    iofile.d
17
#    iofileb.d
18
#    seeksize.d
19
#    iopending
20
#    hotkernel_60
21
#    cpu.d
22
#    dtrace -n 'fbt:zfs::entry { @[probefunc] = count(); }  tick-6s { exit(0); }'
23
#    dtrace -n 'fbt:zfs::entry { @[probefunc] = count(); }  tick-60s { exit(0); }'
24
#
25
# the most telling items I found are the followg:
26
#
27
# #1 The CMD 'zpool-rpool' has 100% of I/O in a system that I don't think 
28
#    should be doing I/O.
29
# 
30
#    lab10# ./psio10 -x 60
31
#           UID   PID %CPU %I/Ot %I/Os %I/Oc %MEM S   TIME CMD
32
#          root     6  0.0  -0.0 100.0 100.0  0.0 S  00:10 zpool-rpool
33
#
34
# #2 The CMD 'hotkerel_60' shows a lot of activity in the last 10 items on
35
#    a system that should be idle paritcularly 28 calls in 60 seconds to
36
#    genunix`fsflush_do_pages - here I show only the LAST 10 lines.
37
#
38
#        FUNCTION                                                COUNT   PCNT
39
#        unix`mutex_enter                                            8   0.0%
40
#        unix`splr                                                   8   0.0%
41
#        unix`do_splx                                                9   0.0%
42
#        unix`tsc_read                                              10   0.0%
43
#        TS`ts_update_list                                          11   0.0%
44
#        genunix`dnlc_purge_vfsp                                    19   0.0%
45
#        genunix`fsflush_do_pages                                   28   0.0%
46
#        unix`page_nextn                                            31   0.0%
47
#        unix`acpi_cpu_cstate                                      414   0.2%
48
#        unix`i86_mwait                                         243513  99.7%
49
#
50
# #3 The CMD 'dtrace -n 'fbt:zfs::entry { @[probefunc] = count(); }  tick-6s { exit(0); }' 
51
#    shows a lot of activity in the last 25 lines (only bottom 25 shown) - I don't
52
#    know what 'ddt_stat_add' is but it is called a lot for a system doing nothing.
53
#
54
#        zio_vdev_io_done                                                491
55
#        zio_vdev_io_start                                               492
56
#        vdev_queue_io_to_issue                                          500
57
#        vdev_stat_update                                                511
58
#        zio_create                                                      511
59
#        zio_destroy                                                     511
60
#        zio_gang_tree_free                                              511
61
#        zio_pop_transforms                                              511
62
#        zio_notify_parent                                               541
63
#        zio_done                                                        544
64
#        dmu_tx_is_syncing                                               574
65
#        vdev_lookup_top                                                 614
66
#        vdev_queue_offset_compare                                       694
67
#        zrl_add_debug                                                   808
68
#        zrl_remove                                                      808
69
#        metaslab_segsize_compare                                        953
70
#        zio_execute                                                    1166
71
#        zio_walk_parents                                               1426
72
#        spa_config_held                                                1471
73
#        metaslab_compare                                               1974
74
#        zio_inherit_child_errors                                       2044
75
#        space_map_seg_compare                                          2090
76
#        zio_wait_for_children                                          3570
77
#        refcount_is_zero                                               9189
78
#        ddt_stat_add                                                  23808
79
#
80
# 4) In 60 seconds there are massive amount of calls to 'umount2' as seen via
81
#    the CMD 'cpu.d' - I don't know why only last 7 lines shown.  In addition
82
# there is a lot of 'forksys' calls too.
83
#        
84
#        getdents64                                                    16320
85
#        pollsys                                                       20063
86
#        open64                                                        25169
87
#        lwp_create                                                    27477
88
#        open                                                          38048
89
#        forksys                                                      421456
90
#        umount2                                                    19804321
91
# ------------------------------------------------------
92

    
93

    
94

    
95

    
96
# -----------------------------------------------------------------------------
97
# script 'psio10' below:
98
#                #!/usr/bin/perl
99
#                #
100
#                # psio10 - a "ps -ef" style tool that prints out disk %I/O. Solaris 10.
101
#                #
102
#                # This is designed to highlight processes that are causing the most disk I/O.
103
#                #  This version of psio uses DTrace (Solaris 10). There are other versions
104
#                #  of psio for older Solaris or the SE Toolkit.
105
#                #
106
#                # NOTE: This is old and nasty - the code needs an overhaul for Solaris 10 FCS.
107
#                #       in the meantime, use iosnoop or prustat instead. Eg,
108
#                #
109
#                #              http://www.brendangregg.com/DTrace/iosnoop
110
#                #
111
#                # 12-Mar-2004, ver 0.71                (check for newer versions)
112
#                #
113
#                #
114
#                # USAGE: psio10 [-efhnx] [seconds]
115
#                #       psio10 [-efhnx] -i infile
116
#                #       psio10 -o outfile [seconds]
117
#                #
118
#                #      psio10             # default "ps -ef" style output, 1 second sample
119
#                #      psio10 5           # sample for 5 seconds
120
#                #      psio10 -e          # event listing (raw and verbose)
121
#                #      psio10 -f          # full device output, print lines per device
122
#                #      psio10 -h          # print usage
123
#                #      psio10 --help      # print full help
124
#                #      psio10 -i infile   # read from infile (a psio dump)
125
#                #      psio10 -n          # numbered output, Time(ms) Size(bytes) and Count
126
#                #      psio10 -o outfile  # write to outfile (create a psio dump)
127
#                #      psio10 -s          # reduced output, PID, %I/O and CMD only
128
#                #      psio10 -x          # extended output, %I/Ot %I/Os %I/Oc %CPU and %MEM
129
#                #
130
#                # To conduct careful analysis first write to an "-o outfile", then run psio
131
#                #  with different options on the "-i infile" (eg, "-x", "-n", "-fn", "-e").
132
#                #
133
#                #
134
#                # FIELDS:
135
#                #      %I/O    %I/O by time taken - duration of disk operation over
136
#                #              available time (most useful field)
137
#                #      %I/Ot   same as above
138
#                #      %I/Os   %I/O by size - number of bytes transferred over total bytes
139
#                #              in sample
140
#                #      %I/Oc   %I/O by count - number of operations over total operations
141
#                #              in sample
142
#                #      IOTIME  Time taken for I/O (ms)
143
#                #      IOSIZE  Size of I/O (bytes)
144
#                #      IOCOUNT Count of I/O (number of operations)
145
#                #      DEVICE  Device number or mount point name, eg "/var".
146
#                #      BLOCK   Block address on disk
147
#                #      INODE   Inode number
148
#                #
149
#                # WARNING: psio may use a large amount of memory if long samples are used
150
#                #  on busy systems.
151
#                #
152
#                # SEE ALSO:    se -DWIDE pea.se        # SE Toolkit
153
#                #              http://www.brendangregg.com/psio.html
154
#                #
155
#                # COPYRIGHT: Copyright (c) 2004 Brendan Gregg.
156
#                #
157
#                #  This program is free software; you can redistribute it and/or
158
#                #  modify it under the terms of the GNU General Public License
159
#                #  as published by the Free Software Foundation; either version 2
160
#                #  of the License, or (at your option) any later version.
161
#                #
162
#                #  This program is distributed in the hope that it will be useful,
163
#                #  but WITHOUT ANY WARRANTY; without even the implied warranty of
164
#                #  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
165
#                #  GNU General Public License for more details.
166
#                #
167
#                #  You should have received a copy of the GNU General Public License
168
#                #  along with this program; if not, write to the Free Software Foundation,
169
#                #  Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
170
#                #
171
#                #  (http://www.gnu.org/copyleft/gpl.html)
172
#                #
173
#                # Author: Brendan Gregg  [Sydney, Australia]
174
#                #
175
#                # Todo:
176
#                #  * Process raw I/O as well as block I/O.
177
#                #  * Add direction (In or Out) to event mode.
178
#                #
179
#                # 12-Mar-2004  Brendan Gregg   Created this, based on original psio.
180
#
181
#
182
#                use Getopt::Std;
183
#
184
#                #
185
#                # --- Default Variables ---
186
#                #
187
#                $period = 1;                           # seconds to sample
188
#                $ENV{PATH} = "/usr/bin:/usr/sbin";     # secure $PATH
189
#                $DEBUG = 0;                            # print debug info
190
#                $BYDEVICE = 0;                         # print by device
191
#                $STYLE = 0;                            # normal "ps -ef" style
192
#                $EVENT = 0;                            # event mode (list all operations)
193
#                $OUTPUT = 0;                           # file output mode
194
#                $INPUT = 0;                            # file input mode
195
#                $Bit = `isainfo -b`;                   # determine bit size
196
#                chomp($Bit);
197
#
198
#
199
#                #
200
#                # --- Command Line Arguments ---
201
#                #
202
#                &Help() if $ARGV[0] eq "--help";
203
#                getopts('efhnsxDi:o:') || &Usage();
204
#                &Usage() if $opt_h;
205
#                $DEBUG = 1 if $opt_D;
206
#                $BYDEVICE = 1 if $opt_f;
207
#                $STYLE = 1 if $opt_x;
208
#                $STYLE = 2 if $opt_s;
209
#                $STYLE = 3 if $opt_n;
210
#                $EVENT = 1 if $opt_e;
211
#                $STYLE = -1 if $opt_e;
212
#                $OUTPUT = 1 if $opt_o;
213
#                $fileout = $opt_o if $OUTPUT;
214
#                $INPUT = 1 if $opt_i;
215
#                $filein = $opt_i if $INPUT;
216
#                $period = $ARGV[0] || $period;
217
#                &Usage() unless $period =~ /^\d*$/;
218
#
219
#
220
#                ### Load device info if needed
221
#                &Load_DeviceInfo() if (($BYDEVICE || $EVENT || $OUTPUT) && (! $INPUT));
222
#
223
#
224
#                #
225
#                # --- Print header line ---
226
#                #
227
#                print "Please wait $period seconds, collecting data...\n" if $DEBUG;
228
#                unless ($OUTPUT) {
229
#                       if ($STYLE == 0) {
230
#                               print "     UID   PID  PPID %I/O    STIME TTY      TIME CMD\n";
231
#                       } elsif ($STYLE == 1) {
232
#                               print "     UID   PID %CPU %I/Ot %I/Os %I/Oc %MEM S   ".
233
#                                "TIME CMD\n";
234
#                       } elsif ($STYLE == 2) {
235
#                               print "   PID %I/O CMD\n";
236
#                       } elsif ($STYLE == 3) {
237
#                               print "     UID   PID  IOTIME    IOSIZE IOCOUNT CMD\n";
238
#                       } elsif ($EVENT) {
239
#                               print "     UID   PID IOTIME IOSIZE BLOCK     DEVICE        ".
240
#                                "INODE CMD\n";
241
#                       }
242
#                }
243
#
244
#
245
#                if ($INPUT) {
246
#                       #
247
#                       # --- Read input from file ---
248
#                       #
249
#                       open(IN,$filein) || die "ERROR10: Can't read infile $filein: $!\n";
250
#
251
#                       $line = <IN>;
252
#                       ($junk,$period) = split(' ',$line);
253
#
254
#                       $delim = 0;
255
#                       while ($line = <IN>) {
256
#                               if ($line =~ /^==========================/) {
257
#                                       $delim++;
258
#                                       next;
259
#                               }
260
#                               if ($delim == 0) {
261
#                                       $ps_all .= $line;
262
#                               } elsif ($delim == 1) {
263
#                                       push(@DTRACE,$line);
264
#                               } elsif ($delim == 2) {
265
#                                       ($key,$value) = split(/:/,$line);
266
#                                       chomp($value);
267
#                                       $DeviceFile{$key} = $value;
268
#                               } elsif ($delim == 3) {
269
#                                       ($key,$value) = split(/:/,$line);
270
#                                       chomp($value);
271
#                                       $MountPoint{$key} = $value;
272
#                               }
273
#                       }
274
#                } else {
275
#                       #
276
#                       # --- Sanity Check ---
277
#                       #
278
#                       if (! -r "/dev/mem") {
279
#                               die "ERROR1: Sorry, you must be root to run this.\n";
280
#                       }
281
#
282
#                       #
283
#                       # --- Generate I/O data from DTrace ---
284
#                       #
285
#                       open(DTRACE,"dtrace -q -n \"
286
#
287
#                          /*
288
#                          **  --- DTrace program to capture I/O ---
289
#                          */
290
#
291
#                          /* Initialise sample interval */
292
#
293
#                          dtrace:::BEGIN { secs = $period; }
294
#                          profile:::tick-1sec { secs--; }
295
#                          profile:::tick-1sec /secs == 0/ { exit(0); }
296
#
297
#                          /* Fetch disk block activity */
298
#
299
#                          fbt:genunix:bdev_strategy:entry
300
#                          {
301
#                               bufp = (buf_t *)arg0;
302
#
303
#                               printf(\\\"%d %s %d %d %d %d %d %d %s\\n\\\",
304
#                                timestamp,probefunc,bufp->b_bcount,bufp->b_edev,
305
#                                bufp->_b_blkno._f,pid,curpsinfo->pr_ppid,
306
#                                curpsinfo->pr_euid,curpsinfo->pr_psargs);
307
#                          }
308
#
309
#                          fbt:genunix:biodone:entry
310
#                          {
311
#                               bufp = (buf_t *)arg0;
312
#                               pagep = (page_t *)bufp->b_pages;
313
#                               vnodep = (int)pagep == 0 ? 0 : (vnode_t *)pagep->p_vnode;
314
#                               vnode =  (int)vnodep == 0 ? 0 : (int)vnodep;
315
#                               inodep = (int)vnodep == 0 ? 0 : (inode_t *)vnodep->v_data;
316
#                               inode =  (int)inodep == 0 ? 0 : inodep->i_number;
317
#
318
#                               printf(\\\"%d %s %d %d %d %x %d\\n\\\",
319
#                                timestamp,probefunc,bufp->b_bcount,bufp->b_edev,
320
#                                bufp->_b_blkno._f,vnode,inode);
321
#                          }
322
#
323
#                       \" |") || die "ERROR2: Can't run dtrace: $!\n is this Solaris 10?\n";
324
#
325
#                       @DTRACE = <DTRACE>;
326
#                       close DTRACE;
327
#
328
#                       #
329
#                       # --- Get ps data ---
330
#                       #
331
#                       $ps_all =
332
#                        `ps -eo pid,ruser,ppid,c,stime,tty,time,rss,vsz,pcpu,pmem,s,args` ||
333
#                               die "ERROR3: Can't run \"ps -eo pid,uid,...\": $!\n";
334
#
335
#                }
336
#
337
#
338
#                #
339
#                # --- Save output file and exit if requested ---
340
#                #
341
#                if ($OUTPUT) {
342
#                       open(OUT,">$fileout") || die "ERROR9: Can't write to $fileout: $!\n";
343
#                       print OUT "period $period\n";
344
#                       print OUT $ps_all;
345
#                       print OUT "="x80,"\n";
346
#                       print OUT @DTRACE;
347
#                       print OUT "="x80,"\n";
348
#                       print OUT $devicefiles;
349
#                       print OUT "="x80,"\n";
350
#                       print OUT $mountpoints;
351
#                       close OUT;
352
#                       exit (0);
353
#                }
354
#
355
#
356
#                #
357
#                # --- Process ps data ---
358
#                #
359
#                foreach $line (split("\n",$ps_all)) {
360
#                       next if $line =~ /^\s*PID/;
361
#                       ($pid,$rest) = split(' ',$line,2);
362
#
363
#                       ### Store in memory
364
#                       $Ps{$pid} = $rest;
365
#                }
366
#
367
#
368
#                #
369
#                # --- Process I/O trace results ---
370
#                #
371
#
372
#                $totalio_time = 0;
373
#                $totalio_size = 0;
374
#                $totalio_count = 0;
375
#
376
#                foreach $line (@DTRACE) {
377
#                       chomp($line);
378
#
379
#                       ### Get data
380
#                       ($elapsed,$probe,$size,$dev,$blk,$rest) = split(' ',$line,6);
381
#                       next if $probe eq "";
382
#                       $elapsed = $elapsed / 1_000_000;        # ns -> ms
383
#
384
#                       #
385
#                       #  Store value - the time between I/O events
386
#                       #  These are usually the times between,
387
#                       #       strategy -> biodone             # block device
388
#                       #
389
#                       if ($probe eq "bdev_strategy") {
390
#
391
#                               ($pid,$ppid,$uid,$args) = split(' ',$rest,4);
392
#
393
#                               $Strategy{"$dev:$blk"} = $pid;
394
#                               $LastDev{"$dev"} = $elapsed;
395
#
396
#                               ### Store process details in case ps -ef dosen't see it
397
#                               $DT_Ps{$pid}{args} = $args;
398
#                               $DT_Ps{$pid}{ppid} = $ppid;
399
#                               $DT_Ps{$pid}{uid} = $uid;
400
#
401
#                       } elsif ($probe eq "biodone") {
402
#
403
#                               if (defined $Strategy{"$dev:$blk"}) {
404
#
405
#                                       ($vnode,$inode) = split(' ',$rest);
406
#
407
#                                       ## $start = $StrategyStart{"$dev:$blk"};
408
#                                       ## $truedelta = $elapsed - $start;
409
#                                       #
410
#                                       #  The above lines of code seem obvious, measuring
411
#                                       #  the time between request and completion - but turns
412
#                                       #  out to be a poor estimation of disk I/O. What can
413
#                                       #  happen is we have several consecutive requests that
414
#                                       #  are then serviced by several consecutive
415
#                                       #  completions (tagged queueing). By counting the
416
#                                       #  deltas between all the requests within the group
417
#                                       #  can over-count the actual service time.
418
#                                       #
419
#                                       #  What is simple (and would be a "last resort") is
420
#                                       #  to use the delta time in this event. This works
421
#                                       #  most of the time - but can give poor results during
422
#                                       #  simultaneous multiple disk access. Eg, a fast
423
#                                       #  disk is accessed while a slow disk as accessed -
424
#                                       #  the delta on the slow disk completions can often
425
#                                       #  be the delta to the last fast disk event - recording
426
#                                       #  smaller than expected times.
427
#                                       #
428
#                                       #  Instead of the above we use the delta time between
429
#                                       #  this event and the last disk event on the device.
430
#                                       #  This gives almost perfect results (we still miss
431
#                                       #  the small time taken to populate the tagged queue).
432
#                                       #
433
#                                       if (defined $LastDev{"$dev"}) {
434
#                                               $truedelta = $elapsed - $LastDev{"$dev"};
435
#                                               delete $LastDev{"$dev"};
436
#                                       } else {
437
#                                               $truedelta = 0;
438
#                                       }
439
#
440
#                                       ### Fetch who really called this
441
#                                       $pid = $Strategy{"$dev:$blk"};
442
#
443
#                                       ### Store I/O time
444
#                                       $Delta{$pid} += $truedelta;
445
#                                       $DeltaDev{$pid}{$dev} += $truedelta;
446
#                                       $totalio_time += $truedelta;
447
#
448
#                                       ### Store I/O size
449
#                                       $Size{$pid} += $size;
450
#                                       $SizeDev{$pid}{$dev} += $size;
451
#                                       $totalio_size += $size;
452
#
453
#                                       ### Store I/O count
454
#                                       $Count{$pid}++;
455
#                                       $CountDev{$pid}{$dev}++;
456
#                                       $totalio_count++;
457
#
458
#                                       ### Store event details
459
#                                       if ($EVENT) {
460
#                                               $Event{$elapsed}{pid} = $pid;
461
#                                               $Event{$elapsed}{size} = $size;
462
#                                               $Event{$elapsed}{dev} = $dev;
463
#                                               $Event{$elapsed}{block} = $blk;
464
#                                               $Event{$elapsed}{delta} = $truedelta;
465
#                                               $Event{$elapsed}{vnode} = $vnode;
466
#                                               $Event{$elapsed}{inode} = $inode;
467
#                                       }
468
#
469
#                                       delete $Strategy{"$dev:$blk"};
470
#                                       print "Stored: PID($pid) Delta($truedelta)" .
471
#                                        " strategy -> biodone\n" if $DEBUG;
472
#
473
#                               }
474
#                               $LastDev{"$dev"} = $elapsed;
475
#                       }
476
#                }
477
#
478
#                ### Prevent divide by zero
479
#                $totalio_time = 1000 if $totalio_time == 0;
480
#                $totalio_size = 1 if $totalio_size == 0;
481
#                $totalio_count = 1 if $totalio_count == 0;
482
#
483
#
484
#                ### Cap total I/O time at 100% - either heavy multiple disk access or
485
#                ###                             sampling errors (didn't sleep exactly)
486
#                $factor = 1;
487
#                $factor = (1000 * $period) / $totalio_time if $totalio_time > (1000 * $period);
488
#
489
#
490
#                #
491
#                # --- Print event data if requested ---
492
#                #
493
#                if ($EVENT) {
494
#                       foreach $event (sort {$a <=> $b} (keys(%Event))) {
495
#
496
#                               ### Fetch event details
497
#                               $pid = $Event{$event}{pid};
498
#                               $size = $Event{$event}{size};
499
#                               $dev = $Event{$event}{dev};
500
#                               $block = $Event{$event}{block};
501
#                               $delta = $Event{$event}{delta};
502
#                               $vnode = $Event{$event}{vnode};
503
#
504
#                               ### Fetch device name
505
#                               $device = &Get_DeviceName($dev);
506
#                               $device =~ s/.*, //;    # full name is too long
507
#
508
#                               ### Get process data
509
#                               $line = $Ps{$pid};
510
#                               if ($line eq "") {
511
#                                       ($ruser,$ppid,$c,$stime,$tty,$time,$rss,$vsz,$pcpu,
512
#                                        $pmem,$s,$args) = qw(? ? ? ? ? ? ? ? ? ? ? ?);
513
#
514
#                                       ### Check if DTrace caught these details
515
#                                       if (defined $DT_Ps{$pid}{uid}) {
516
#                                               $uid = $DT_Ps{$pid}{uid};
517
#                                               $ppid = $DT_Ps{$pid}{ppid};
518
#                                               $args = $DT_Ps{$pid}{args};
519
#                                               @Fields = getpwuid($uid);
520
#                                               $ruser = $Fields[0];
521
#                                       }
522
#                               } else {
523
#                                       ($ruser,$ppid,$c,$stime,$tty,$time,$rss,$vsz,$pcpu,
524
#                                        $pmem,$s,$args) = split(' ',$line,12);
525
#                               }
526
#
527
#                               ### Print formatted output
528
#                               printf("%8s %5s %6.0f %6s %-9s %-12s %6s %s\n",$ruser,$pid,
529
#                                $delta,$size,$block,$device,$inode,$args);
530
#                       }
531
#                       exit (0);
532
#                }
533
#
534
#
535
#                #
536
#                # --- Print ps data with I/O ---
537
#                #
538
#                foreach $pid (sort {$Delta{$b} <=> $Delta{$a}} (keys(%Delta))) {
539
#                       $delta = $Delta{$pid};
540
#                       $size = $Size{$pid};
541
#                       $count = $Count{$pid};
542
#
543
#                       #
544
#                       #  Calculate percentages
545
#                       #
546
#                       $pcntio_time = $factor * $delta / (10 * $period); #/10 is ms -> %
547
#                       $pcntio_time = sprintf("%.1f",$pcntio_time);
548
#                       $pcntio_size = 100 * $size / $totalio_size;       # *100 makes it %
549
#                       $pcntio_size = sprintf("%.1f",$pcntio_size);
550
#                       $pcntio_count = 100 * $count / $totalio_count;    # *100 makes it %
551
#                       $pcntio_count = sprintf("%.1f",$pcntio_count);
552
#
553
#                       ### Get process data
554
#                       $line = $Ps{$pid};
555
#
556
#                       #
557
#                       #  Some processes will begin and end too quickly - they
558
#                       #  will be seen by DTrace but not during the ps -ef sample,
559
#                       #  in which case we use the details from DTrace instead of ps.
560
#                       #
561
#                       if ($line eq "") {
562
#                               ($ruser,$ppid,$c,$stime,$tty,$time,$rss,$vsz,$pcpu,$pmem,$s,
563
#                                $args) = qw(? ? ? ? ? ? ? ? ? ? ?);
564
#
565
#                               ### Check if DTrace caught these details
566
#                               if (defined $DT_Ps{$pid}{uid}) {
567
#                                       $uid = $DT_Ps{$pid}{uid};
568
#                                       $ppid = $DT_Ps{$pid}{ppid};
569
#                                       $args = $DT_Ps{$pid}{args};
570
#                                       @Fields = getpwuid($uid);
571
#                                       $ruser = $Fields[0];
572
#                               }
573
#                       } else {
574
#                               ($ruser,$ppid,$c,$stime,$tty,$time,$rss,$vsz,$pcpu,$pmem,$s,
575
#                                $args) = split(' ',$line,12);
576
#                               $stime =~ tr/_/ /;
577
#                       }
578
#
579
#                       #
580
#                       #  Format and print output
581
#                       #
582
#                       if ($STYLE == 0) {
583
#                               printf("%8s %5s %5s %4s %8s %-6s %6s %s\n",$ruser,$pid,$ppid,
584
#                                $pcntio_time,$stime,$tty,$time,$args);
585
#                       } elsif ($STYLE == 1) {
586
#                               printf("%8s %5s %4s %5s %5s %5s %4s %1s %6s %s\n",$ruser,$pid,
587
#                                $pcpu,$pcntio_time,$pcntio_size,$pcntio_count,$pmem,$s,
588
#                                $time,$args);
589
#                       } elsif ($STYLE == 2) {
590
#                               printf("%6s %4s %s\n",$pid,$pcntio_time,$args);
591
#                       } elsif ($STYLE == 3) {
592
#                               printf("%8s %5s %7.0f %9s %7s %s\n",$ruser,$pid,$delta,
593
#                                $size,$count,$args);
594
#                       }
595
#
596
#                       #
597
#                       #  Now print results on a device by device basis, if requested
598
#                       #
599
#                       if ($BYDEVICE) {
600
#                          foreach $dev (sort {$DeltaDev{$pid}{$b} <=>
601
#                           $DeltaDev{$pid}{$a}} (keys(%{$DeltaDev{$pid}}))) {
602
#
603
#                               ### Calculate percentages
604
#                               $delta = $DeltaDev{$pid}{$dev};
605
#                               $size = $SizeDev{$pid}{$dev};
606
#                               $count = $CountDev{$pid}{$dev};
607
#                               $pcntio_time = $factor * $delta / (10 * $period);
608
#                               $pcntio_time = sprintf("%.1f",$pcntio_time);
609
#                               $pcntio_size = 100 * $size / $totalio_size;
610
#                               $pcntio_size = sprintf("%.1f",$pcntio_size);
611
#                               $pcntio_count = 100 * $count / $totalio_count;
612
#                               $pcntio_count = sprintf("%.1f",$pcntio_count);
613
#
614
#                               ### Fetch device name
615
#                               $device = &Get_DeviceName($dev);
616
#
617
#                               ### Format and print output
618
#                               if ($STYLE == 0) {
619
#                                       printf("%8s %5s %5s %4s  %s\n",'"','"','"',
620
#                                        $pcntio_time,$device);
621
#                               } elsif ($STYLE == 1) {
622
#                                       printf("%8s %5s %4s %5s %5s %5s   %s\n",'"','"',
623
#                                       '"',$pcntio_time,$pcntio_size,$pcntio_count,$device);
624
#                               } elsif ($STYLE == 2) {
625
#                                       printf("%6s %4s  %s\n",'"',$pcntio_time,$device);
626
#                               } elsif ($STYLE == 3) {
627
#                                       printf("%8s %5s %7.0f %9s %7s  %s\n",'"','"',
628
#                                        $delta,$size,$count,$device);
629
#                               }
630
#                          }
631
#                       }
632
#
633
#
634
#                       delete $Ps{$pid};
635
#                }
636
#
637
#                #
638
#                # --- Print leftover ps lines (%0) ---
639
#                #
640
#                foreach $pid (sort {$a <=> $b} (keys(%Ps))) {
641
#                       $line = $Ps{$pid};
642
#
643
#                       ($ruser,$ppid,$c,$stime,$tty,$time,$rss,$vsz,$pcpu,$pmem,$s,$args) =
644
#                        split(' ',$line,12);
645
#                       $stime =~ tr/_/ /;
646
#
647
#                       ### Format and print output
648
#                       if ($STYLE == 0) {
649
#                               printf("%8s %5s %5s %4s %8s %-6s %6s %s\n",$ruser,$pid,$ppid,
650
#                                "0.0",$stime,$tty,$time,$args);
651
#                       } elsif ($STYLE == 1) {
652
#                               printf("%8s %5s %4s %5s %5s %5s %4s %1s %6s %s\n",$ruser,$pid,
653
#                                $pcpu,"0.0","0.0","0.0",$pmem,$s,$time,$args);
654
#                       } elsif ($STYLE == 2) {
655
#                               printf("%6s %4s %s\n",$pid,"0.0",$args);
656
#                       } elsif ($STYLE == 3) {
657
#                               printf("%8s %5s %7s %9s %7s %s\n",$ruser,$pid,0,0,0,$args);
658
#                       }
659
#
660
#                }
661
#
662
#
663
#
664
#                #########################
665
#                # --- SUBROUTINES ---
666
#                #
667
#
668
#
669
#                # Load_DeviceInfo - Loads general device info for devicename lookups
670
#                #
671
#                sub Load_DeviceInfo {
672
#                       #
673
#                       #  Store device number to device filename lookup in %DeviceFile
674
#                       #
675
#                       foreach $file (</dev/dsk/*>) {
676
#                               @Stat = stat($file);
677
#                               if ($Bit == 64) {
678
#                                       ($major,$minor) = unpack('nn',pack('N',$Stat[6]));
679
#                                       $major *= 2**30;
680
#                                       $filedev = $major + $minor;
681
#                               } else {
682
#                                       $filedev = $Stat[6];
683
#                               }
684
#                               $DeviceFile{$filedev} = $file;
685
#                               $devicefiles .= "$filedev:$file\n";     # for output file
686
#                       }
687
#
688
#                       #
689
#                       #  Get mount point info in %MountPoint
690
#                       #
691
#                       open(MNTTAB,"/etc/mnttab") ||
692
#                               die "ERROR8: Can't read /etc/mnttab: $!\n";
693
#
694
#                       while ($line = <MNTTAB>) {
695
#                               ($fs,$mount,$rest) = split(' ',$line,3);
696
#                               $MountPoint{$fs} = $mount;
697
#                               $mountpoints .= "$fs:$mount\n";         # for output file
698
#                       }
699
#                       close MNTTAB;
700
#                }
701
#
702
#
703
#                # Get_DeviceName - Gets the mount point or block device name from the
704
#                #              extended device number
705
#                #
706
#                sub Get_DeviceName {
707
#                       my $dev = shift;
708
#                       my ($line,$rest,$file,$mount);
709
#
710
#                       ### Quick fetch if already known
711
#                       if (defined $DeviceName{$dev}) { return $DeviceName{$dev}; }
712
#
713
#                       #
714
#                       #  Generate device name
715
#                       #
716
#                       $file = $DeviceFile{$dev};
717
#                       if (defined $file) {
718
#                               $DeviceName{$dev} = $file;
719
#                               if (defined $MountPoint{$file}) {
720
#                                       $DeviceName{$dev} .= ", $MountPoint{$file}";
721
#                               }
722
#                       } else {
723
#                               $DeviceName{$dev} = $dev;
724
#                       }
725
#
726
#                       ### Return
727
#                       return $DeviceName{$dev};
728
#                }
729
#
730
#
731
#                # Usage - print a usage message and exit.
732
#                #
733
#                sub Usage {
734
#                       print STDERR <<END;
735
#                psio10 ver 0.71
736
#                USAGE: psio10 [-efhmnx] [seconds]
737
#                       psio10 [-efhnx] -i infile
738
#                       psio10 -o outfile [seconds]
739
#                   eg,
740
#                      psio10 5           # 5 second sample
741
#                      psio10 -x          # extended output, %I/Ot %I/Os %I/Oc %CPU and %MEM
742
#                      psio10 -e          # event listing (raw and verbose)
743
#                      psio10 --help      # print full help
744
#                END
745
#                       exit (1);
746
#                }
747
#
748
#
749
#                # Help - print help. Actually strip it from the comments
750
#                #              at the top of the code.
751
#                #
752
#                sub Help {
753
#                       open (MYSELF,"$0") || die "ERROR8: I can't see myself: $!\n";
754
#                       @Myself = <MYSELF>;
755
#                       close MYSELF;
756
#
757
#                       ### Print comment from top of code
758
#                       foreach $line (@Myself) {
759
#                               last if $line !~ /^#/;
760
#                               last if $line =~ /^# Todo:/;
761
#                               next if $line =~ m:^#!/usr/bin/perl:;
762
#                               $line =~ s/^# //;
763
#                               $line =~ s/^#//;
764
#                               print $line;
765
#                       }
766
#                       print "\n";
767
#
768
#                       exit(0);
769
#                }
770
#
771
# -----------------------------------------------------------------------------
772
# script output/run for ./psio10 -x 60
773
# ----------------------------
774
lab10# ./psio10 -x 60
775
     UID   PID %CPU %I/Ot %I/Os %I/Oc %MEM S   TIME CMD
776
    root     6  0.0  -0.0 100.0 100.0  0.0 S  00:10 zpool-rpool
777
    root     0  0.0   0.0   0.0   0.0  0.0 T  00:01 sched
778
    root     1  0.0   0.0   0.0   0.0  0.0 S  00:00 /sbin/init
779
    root     2  0.0   0.0   0.0   0.0  0.0 S  00:00 pageout
780
    root     3  0.1   0.0   0.0   0.0  0.0 S  03:00 fsflush
781
    root     4  0.0   0.0   0.0   0.0  0.0 S  00:00 kcfpoold
782
    root 100003  0.0   0.0   0.0   0.0  0.1 S  00:02 /lib/svc/bin/svc.startd
783
    root 100005  0.0   0.0   0.0   0.0  0.1 S  00:06 /lib/svc/bin/svc.configd
784
  netadm 100038  0.0   0.0   0.0   0.0  0.1 S  00:00 /lib/inet/ipmgmtd
785
    root 100041  0.0   0.0   0.0   0.0  0.0 S  00:00 /sbin/dlmgmtd
786
  netcfg 100043  0.0   0.0   0.0   0.0  0.0 S  00:00 /lib/inet/netcfgd
787
    root 100150  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/pfexecd
788
    root 100200  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/zones/zonestatd
789
    root 100204  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/dbus-daemon --system
790
    root 100220  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/lib/sysevent/syseventd
791
    root 100232  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/power/powerd
792
    root 100263  0.0   0.0   0.0   0.0  0.1 S  00:02 devfsadmd
793
    root 100294  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/sbin/cupsd -C /etc/cups/cupsd.conf
794
    root 100295  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/lib/picl/picld
795
    root 100325  0.0   0.0   0.0   0.0  0.0 S  00:00 /lib/svc/method/iscsid
796
    root 100342  0.0   0.0   0.0   0.0  0.1 S  00:04 /usr/sbin/nscd
797
    root 100372  0.0   0.0   0.0   0.0  0.1 S  00:04 /usr/lib/hal/hald --daemon=yes
798
    root 100376  0.0   0.0   0.0   0.0  0.0 S  00:00 hald-runner
799
    root 100392  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/hal/hald-addon-network-discovery
800
    root 100394  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/hal/hald-addon-cpufreq
801
    root 100395  0.0   0.0   0.0   0.0  0.0 S  00:03 /usr/lib/hal/hald-addon-acpi
802
    root 100427  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/sbin/console-kit-daemon
803
    root 100440  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/sbin/cron
804
    root 100498  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/rmvolmgr -s
805
  daemon 100673  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/sbin/rpcbind
806
    root 100694  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/autofs/automountd
807
    root 100697  0.0   0.0   0.0   0.0  0.0 S  00:09 /usr/lib/autofs/automountd
808
    root 100708  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/saf/sac -t 300
809
    root 100719  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/inet/in.ndpd
810
    root 100720  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/saf/ttymon
811
    root 100725  0.0   0.0   0.0   0.0  0.1 S  00:01 /usr/lib/inet/inetd start
812
    root 100733  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/utmpd
813
    root 100759  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -h -p lab1
814
    root 100768  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/sbin/syslogd
815
    root 100801  0.0   0.0   0.0   0.0  0.2 S  00:05 /usr/lib/fm/fmd/fmd
816
   smmsp 100810  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/sendmail -Ac -q15m
817
    root 100859  0.0   0.0   0.0   0.0  0.0 S  00:00 /usr/lib/ssh/sshd
818
    root 100900  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/lib/rad/rad -sp
819
    root 100915  0.0   0.0   0.0   0.0  0.1 S  00:11 /usr/perl5/bin/perl /usr/lib/intrd
820
    root 100921  0.0   0.0   0.0   0.0  0.0 S  00:03 /usr/sbin/in.routed
821
    root 101072  0.0   0.0   0.0   0.0  0.0 S  00:03 /usr/lib/sendmail -bl -q15m
822
    root 101074  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/lib/ssh/sshd
823
   admin 101075  0.0   0.0   0.0   0.0  0.1 S  00:02 /usr/lib/ssh/sshd
824
   admin 101080  0.0   0.0   0.0   0.0  0.1 S  00:00 -bash
825
    root 101090  0.0   0.0   0.0   0.0  0.1 S  00:00 -bash
826
    root 101752  0.0   0.0   0.0   0.0  0.0 S  00:00 csh
827
    root 102348  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/lib/ssh/sshd
828
   admin 102349  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/lib/ssh/sshd
829
   admin 102354  0.0   0.0   0.0   0.0  0.1 S  00:00 -bash
830
    root 102378  0.0   0.0   0.0   0.0  0.1 S  00:00 bash
831
    root 104631  0.0   0.0   0.0   0.0  0.0 S  00:00 /bin/csh -f ./do_disk.csh
832
    root 104636  0.0   0.0   0.0   0.0  0.1 S  00:00 /usr/bin/perl ./psio10 -x 60
833
    root 104639  0.0   0.0   0.0   0.0  0.0 O  00:00 ps -eo pid,ruser,ppid,c,stime,tty,time,rss,vsz,pcpu,pmem,s,args
834

    
835

    
836
# -----------------------------------------------------------------------------
837
# script 'fsflush.d' below:
838
#                #!/usr/sbin/dtrace -s
839
#
840
#                #pragma D option quiet
841
#
842
#                BEGIN
843
#                {
844
#                    lexam = 0; lscan = 0; llock = 0; lmod = 0; lcoal = 0; lrel = 0; ltime = 0;
845
#                    printf("Tracing... Hit Ctrl-C to end or wait 60 seconds.\n");
846
#                    printf("%10s %10s %10s %10s %10s %10s %10s\n", "SCANNED", "EXAMINED",
847
#                        "LOCKED", "MODIFIED", "COALESCE", "RELEASES", "TIME(ns)");
848
#                }
849
#
850
#                tick-1s
851
#                /lexam/
852
#                {
853
#                    printf("%10d %10d %10d %10d %10d %10d %10d\n", `fsf_total.fsf_scan - lscan,
854
#                        `fsf_total.fsf_examined - lexam, `fsf_total.fsf_locked - llock,
855
#                        `fsf_total.fsf_modified - lmod, `fsf_total.fsf_coalesce - lcoal,
856
#                        `fsf_total.fsf_releases - lrel, `fsf_total.fsf_time - ltime);
857
#                    lexam = `fsf_total.fsf_examined;
858
#                    lscan = `fsf_total.fsf_scan;
859
#                    llock = `fsf_total.fsf_locked;
860
#                    lmod = `fsf_total.fsf_modified;
861
#                    lcoal = `fsf_total.fsf_coalesce;
862
#                    lrel = `fsf_total.fsf_releases;
863
#                    ltime = `fsf_total.fsf_time;
864
#                }
865
#
866
#                /*
867
#                 * First time through
868
#                 */
869
#
870
#                tick-1s
871
#                /!lexam/
872
#                {
873
#                    lexam = `fsf_total.fsf_examined;
874
#                    lscan = `fsf_total.fsf_scan;
875
#                    llock = `fsf_total.fsf_locked;
876
#                    lmod = `fsf_total.fsf_modified;
877
#                    lcoal = `fsf_total.fsf_coalesce;
878
#                    ltime = `fsf_total.fsf_time;
879
#                    lrel = `fsf_total.fsf_releases;
880
#                }
881
#
882
#                tick-60s
883
#                {
884
#                        exit(0);
885
#                }
886
#
887
# -----------------------------------------------------------------------------
888
# script output/run for ./fsflush.d
889
# ----------------------------
890
lab10# ./fsflush.d
891
Tracing... Hit Ctrl-C to end or wait 60 seconds.
892
   SCANNED   EXAMINED     LOCKED   MODIFIED   COALESCE   RELEASES   TIME(ns)
893
    139623        274          0          0          0          0      27379
894
    139623        274          0          0          0          0      27739
895
    139623        274          0          0          0          0      27466
896
    139623        274          0          0          0          0      30791
897
    139623       2739          0          0          0          0      90644
898
    139623        274          0          0          0          0      28092
899
    139623        274          0          0          0          0      28090
900
    139623        274          0          0          0          0      27385
901
    139623        274          0          0          0          0      27764
902
    139623        274          0          0          0          0      27311
903
    139623        274          0          0          0          0      27436
904
    139623        274          0          0          0          0      27481
905
    139623        274          0          0          0          0      27376
906
    139623        274          0          0          0          0      27282
907
    139623        274          0          0          0          0      27621
908
    139623        274          0          0          0          0      27658
909
    139623        274          0          0          0          0      27749
910
    139623        274          0          0          0          0      27844
911
    139623        274          0          0          0          0      27325
912
    139623        274          0          0          0          0      27766
913
    139623        274          0          0          0          0      27576
914
    139623        274          0          0          0          0      27719
915
    139623        274          0          0          0          0      27864
916
    139623      30270         50          1          0         49    1071603
917
    139623     139623         90         16          0         24    3626488
918
    139623     139623          0          0          0          0    3128993
919
    139623     139623         16          3          0          1    3212043
920
    139623     139623       3406         57          0         21    4129617
921
    139623     113592       1636         56          0         10    2968712
922
    139623        274          0          0          0          0      30601
923
    139623        274          0          0          0          0      26882
924
    139623        274          0          0          0          0      27305
925
    139623        274          0          0          0          0      27501
926
    139623        274          0          0          0          0      30232
927
    139623       2739          0          0          0          0      91105
928
    139623        274          0          0          0          0      27498
929
    139623        274          0          0          0          0      27789
930
    139623        274          0          0          0          0      27330
931
    139623        274          0          0          0          0      27507
932
    139623        274          0          0          0          0      27268
933
    139623        274          0          0          0          0      27937
934
    139623        274          0          0          0          0      27268
935
    139623        274          0          0          0          0      27268
936
    139623        274          0          0          0          0      27746
937
    139623        274          0          0          0          0      27558
938
    139623        274          0          0          0          0      27880
939
    139623        274          0          0          0          0      27659
940
    139623        274          0          0          0          0      27664
941
    139623        274          0          0          0          0      27714
942
    139623        274          0          0          0          0      27855
943
    139623        274          0          0          0          0      27330
944
    139623        274          0          0          0          0      27687
945
    139623        274          0          0          0          0      28042
946
    139623      33847          2          2          0          0    1131105
947
    139623     139623         65         15          0          0    3536535
948
    139623     139623          2          0          0          0    3132767
949
    139623     139623         21          3          0          0    3211807
950
    139623     139623       3386         57          0          0    4086934
951
    139623     110015       1617         56          0          0    2865051
952

    
953

    
954

    
955
# -----------------------------------------------------------------------------
956
# script 'fspaging.d' below:
957
#                #!/usr/sbin/dtrace -s
958
#
959
#                #pragma D option quiet
960
#
961
#
962
#                :::BEGIN
963
#                {
964
#                        printf("Tracing... Hit Ctrl-C to end or wait 60 seconds.\n");
965
#                       printf("%-10s %-10s %51s %2s %8s %8s\n",
966
#                               "Event", "Device", "Path", "RW", "Size", "Offset");
967
#                }
968
#
969
#                ::bwrite_common:entry
970
#                {
971
#                       self->bwrite = 1;
972
#                }
973
#
974
#                ::bwrite_common:return
975
#                {
976
#                       self->bwrite = 0;
977
#                }
978
#
979
#                ::bread_common:entry
980
#                {
981
#                       self->bread = 1;
982
#                }
983
#
984
#                ::bread_common:return
985
#                {
986
#                       self->bread = 0;
987
#                }
988
#
989
#                ::fop_putpage:entry
990
#                /self->trace == 0 && (((vnode_t *)arg0)->v_path)/
991
#                {
992
#                       vp = (vnode_t*)arg0;
993
#                       b_addr = arg1;
994
#                       len = arg2;
995
#                       self->path=stringof(vp->v_path);
996
#                       printf("put-page       %58s    %8d\n", self->path, len);
997
#                       self->type = "putpage-io";
998
#                       self->trace = 1;
999
#                }
1000
#
1001
#                ::fop_putpage:return
1002
#                /self->trace == 1/
1003
#                {
1004
#                       self->trace = 0;
1005
#                }
1006
#
1007
#                ::fop_getpage:entry
1008
#                /self->trace == 0 && (((vnode_t *)arg0)->v_path)/
1009
#                {
1010
#                       vp = (vnode_t*)arg0;
1011
#                       b_addr = arg1;
1012
#                       len = arg2;
1013
#                       self->path=stringof(vp->v_path);
1014
#                       printf("get-page       %58s    %8d\n", self->path, len);
1015
#                       self->type = "getpage-io";
1016
#                       self->trace = 1;
1017
#                }
1018
#
1019
#                ::fop_getpage:return
1020
#                /self->trace == 1/
1021
#                {
1022
#                       self->trace = 0;
1023
#                }
1024
#
1025
#                io::bdev_strategy:start
1026
#                /self->trace/
1027
#                {
1028
#                       printf("%s %10s %51s %2s %8d %8u\n", self->type, args[1]->dev_statname,
1029
#                           self->path, args[0]->b_flags & B_READ ? "R" : "W",
1030
#                           args[0]->b_bcount, args[0]->b_blkno);
1031
#                }
1032
#
1033
#                io::bdev_strategy:start
1034
#                /self->trace == 0 && (self->bwrite || self->bread)/
1035
#                {
1036
#                       printf("buffer-io  %10s %51s %2s %8d %8u\n", args[1]->dev_statname,
1037
#                           args[2]->fi_pathname, args[0]->b_flags & B_READ ? "R" : "W",
1038
#                           args[0]->b_bcount, args[0]->b_blkno);
1039
#                }
1040
#
1041
#                io::bdev_strategy:start
1042
#                /self->trace == 0 && (!(self->bwrite || self->bread))/
1043
#                {
1044
#                       printf("other-io   %10s %51s %2s %8d %8u\n", args[1]->dev_statname,
1045
#                           args[2]->fi_pathname, args[0]->b_flags & B_READ ? "R" : "W",
1046
#                           args[0]->b_bcount, args[0]->b_blkno);
1047
#                }
1048
#
1049
#                tick-60s
1050
#                {
1051
#                        exit(0);
1052
#                }
1053
#
1054
# -----------------------------------------------------------------------------
1055
# script output/run for ./fspaging.d
1056
# ----------------------------
1057
lab10# ./fspaging.d
1058
Tracing... Hit Ctrl-C to end or wait 60 seconds.
1059
Event      Device                                                    Path RW     Size   Offset
1060
get-page                                    /usr/lib/amd64/libdtrace.so.1        4096
1061
get-page                                             /lib/amd64/libc.so.1        4096
1062
get-page                                             /lib/amd64/libc.so.1        4096
1063
get-page                                    /usr/lib/amd64/libdtrace.so.1        4096
1064
other-io          sd2                                              <none>  W      512 201487849
1065
other-io          sd3                                              <none>  W      512 201487849
1066
other-io          sd2                                              <none>  W      512 134347860
1067
other-io          sd3                                              <none>  W      512 134347860
1068
other-io          sd2                                              <none>  W     1024 201469925
1069
other-io          sd3                                              <none>  W     1024 201469925
1070
other-io          sd2                                              <none>  W     1024 134327668
1071
other-io          sd3                                              <none>  W     1024 134327668
1072
other-io          sd2                                              <none>  W    13312 134327642
1073
other-io          sd3                                              <none>  W    13312 134327642
1074
other-io          sd2                                              <none>  W     2048 134281899
1075
other-io          sd2                                              <none>  W     4096 163666884
1076
other-io          sd3                                              <none>  W     2048 134281899
1077
other-io          sd3                                              <none>  W     6144 163640312
1078
other-io          sd2                                              <none>  W     3584 201487875
1079
other-io          sd3                                              <none>  W     3584 201487875
1080
other-io          sd3                                              <none>  W     2048 134281903
1081
other-io          sd3                                              <none>  W     1024 134329146
1082
other-io          sd2                                              <none>  W     1024 134329146
1083
other-io          sd2                                              <none>  W     5632 134347875
1084
other-io          sd2                                              <none>  W     1024 163640324
1085
other-io          sd2                                              <none>  W     4096 201447447
1086
other-io          sd2                                              <none>  W     1024 201469939
1087
other-io          sd2                                              <none>  W     3584 201487882
1088
other-io          sd3                                              <none>  W     8704 134347883
1089
other-io          sd3                                              <none>  W     1024 163640324
1090
other-io          sd3                                              <none>  W     4096 201447447
1091
other-io          sd2                                              <none>  W     1536 201487864
1092
other-io          sd3                                              <none>  W     1536 201487864
1093
other-io          sd2                                              <none>  W     1536 134347875
1094
other-io          sd3                                              <none>  W     1536 134347875
1095
other-io          sd2                                              <none>  W     1536 163666895
1096
other-io          sd2                                              <none>  W     2048 134281899
1097
other-io          sd2                                              <none>  W     2048 163635760
1098
other-io          sd2                                              <none>  W      512 163666891
1099
other-io          sd3                                              <none>  W     1024 163640324
1100
other-io          sd2                                              <none>  W     7680 163666898
1101
other-io          sd2                                              <none>  W     1024 201469939
1102
other-io          sd3                                              <none>  W     2048 201487860
1103
other-io          sd2                                              <none>  W     4096 201489434
1104
other-io          sd3                                              <none>  W    11264 201487867
1105
other-io          sd2                                              <none>  W     1024      492
1106
other-io          sd2                                              <none>  W     1024     1004
1107
other-io          sd2                                              <none>  W     1024 234275308
1108
other-io          sd2                                              <none>  W     1024 234275820
1109
other-io          sd3                                              <none>  W     1024      492
1110
other-io          sd3                                              <none>  W     1024     1004
1111
other-io          sd3                                              <none>  W     1024 234275308
1112
other-io          sd3                                              <none>  W     1024 234275820
1113
other-io          sd2                                              <none>  W     8192 201339377
1114
other-io          sd3                                              <none>  W     8192 201339377
1115
other-io          sd2                                              <none>  W      512 201487852
1116
other-io          sd3                                              <none>  W      512 201487852
1117
other-io          sd2                                              <none>  W      512 134347863
1118
other-io          sd3                                              <none>  W      512 134347863
1119
other-io          sd2                                              <none>  W      512 201487851
1120
other-io          sd3                                              <none>  W      512 201487851
1121
other-io          sd2                                              <none>  W      512 134347862
1122
other-io          sd3                                              <none>  W      512 134347862
1123
other-io          sd2                                              <none>  W     2048 163635760
1124
other-io          sd2                                              <none>  W     2048 201447447
1125
other-io          sd3                                              <none>  W     4096 134347864
1126
other-io          sd3                                              <none>  W     4096 201487853
1127
other-io          sd2                                              <none>  W     3072 201469927
1128
other-io          sd3                                              <none>  W     3072 201469927
1129
other-io          sd2                                              <none>  W     3072 134329122
1130
other-io          sd3                                              <none>  W     3072 134329122
1131
other-io          sd2                                              <none>  W     3072 163640312
1132
other-io          sd2                                              <none>  W     4096 201489434
1133
other-io          sd2                                              <none>  W     3584 134347886
1134
other-io          sd2                                              <none>  W     2048 134281903
1135
other-io          sd2                                              <none>  W     4096 134310396
1136
other-io          sd2                                              <none>  W      512 134347871
1137
other-io          sd2                                              <none>  W     4096 163635760
1138
other-io          sd2                                              <none>  W     9216 163666895
1139
other-io          sd2                                              <none>  W     1536 134347872
1140
other-io          sd3                                              <none>  W     2048 134281899
1141
other-io          sd3                                              <none>  W     4096 163635760
1142
other-io          sd3                                              <none>  W     7680 163666961
1143
other-io          sd2                                              <none>  W     2048 201447451
1144
other-io          sd3                                              <none>  W     2048 201447451
1145
other-io          sd2                                              <none>  W     1024 134329146
1146
other-io          sd2                                              <none>  W     1536 134347872
1147
other-io          sd2                                              <none>  W     2048 163635764
1148
other-io          sd2                                              <none>  W     1536 163666892
1149
other-io          sd3                                              <none>  W    11264 134347878
1150
other-io          sd3                                              <none>  W     3072 134329122
1151
other-io          sd2                                              <none>  W      512 134347871
1152
other-io          sd3                                              <none>  W     4096 163635760
1153
other-io          sd3                                              <none>  W    11264 163666891
1154
other-io          sd2                                              <none>  W     7680 163666961
1155
other-io          sd2                                              <none>  W     3072 201469927
1156
other-io          sd2                                              <none>  W     2048 201487860
1157
other-io          sd2                                              <none>  W    11264 201487867
1158
other-io          sd3                                              <none>  W     4096 201489434
1159
other-io          sd2                                              <none>  W     8192 134265647
1160
other-io          sd2                                              <none>  W    13312 201469899
1161
other-io          sd3                                              <none>  W    13312 201469899
1162
other-io          sd2                                              <none>  W      512 201487850
1163
other-io          sd3                                              <none>  W      512 201487850
1164
other-io          sd2                                              <none>  W      512 134347861
1165
other-io          sd3                                              <none>  W      512 134347861
1166
other-io          sd2                                              <none>  W     3072 201469927
1167
other-io          sd3                                              <none>  W     3072 201469927
1168
other-io          sd2                                              <none>  W     3072 134329122
1169
other-io          sd3                                              <none>  W     3072 134329122
1170
other-io          sd2                                              <none>  W     6144 163640312
1171
other-io          sd2                                              <none>  W     4096 201487853
1172
other-io          sd3                                              <none>  W     4096 163666884
1173
other-io          sd3                                              <none>  W     4096 201489434
1174
other-io          sd3                                              <none>  W     4096 134310396
1175
other-io          sd3                                              <none>  W     4096 134347875
1176
other-io          sd2                                              <none>  W     2048 134281899
1177
other-io          sd2                                              <none>  W     3584 134347893
1178
other-io          sd2                                              <none>  W      512 163666891
1179
other-io          sd2                                              <none>  W     7680 163666961
1180
other-io          sd2                                              <none>  W     1536 163666892
1181
other-io          sd2                                              <none>  W     7680 201487860
1182
other-io          sd3                                              <none>  W     2048 134347871
1183
other-io          sd3                                              <none>  W     3072 163640312
1184
other-io          sd3                                              <none>  W    11264 163666891
1185
other-io          sd3                                              <none>  W     1024 201469939
1186
other-io          sd3                                              <none>  W     7680 201487860
1187
other-io          sd3                                              <none>  W     1024 201469939
1188
other-io          sd3                                              <none>  W     2048 134281903
1189
other-io          sd3                                              <none>  W     1024 134329146
1190
other-io          sd2                                              <none>  W    11264 134347878
1191
other-io          sd2                                              <none>  W     1024 163640324
1192
other-io          sd3                                              <none>  W      512 134347871
1193
other-io          sd3                                              <none>  W     2048 134281899
1194
other-io          sd2                                              <none>  W     3072 134329122
1195
other-io          sd2                                              <none>  W     3072 163640312
1196
other-io          sd3                                              <none>  W     3072 163640312
1197
other-io          sd3                                              <none>  W     7680 163666961
1198
other-io          sd2                                              <none>  W     2048 201447447
1199
other-io          sd3                                              <none>  W     2048 201447447
1200
other-io          sd3                                              <none>  W     3072 201469927
1201
other-io          sd3                                              <none>  W     8192 134265647
1202
other-io          sd2                                              <none>  W     3072 201469933
1203
other-io          sd3                                              <none>  W     3072 201469933
1204
other-io          sd2                                              <none>  W     3072 134329140
1205
other-io          sd3                                              <none>  W     3072 134329140
1206
other-io          sd2                                              <none>  W     4096 134347864
1207
other-io          sd3                                              <none>  W     2048 163635760
1208
other-io          sd3                                              <none>  W     2048 201447447
1209
other-io          sd3                                              <none>  W     3584 201487882
1210
other-io          sd2                                              <none>  W     2048 134281903
1211
other-io          sd2                                              <none>  W     4096 134310396
1212
other-io          sd3                                              <none>  W     4096 134310396
1213
other-io          sd3                                              <none>  W     1536 134347872
1214
put-page                                                         /tmp/y.i        4096
1215
put-page                                                         /tmp/y.i        4096
1216
put-page                                                         /tmp/y.i        4096
1217
put-page                                                         /tmp/t.t        4096
1218
put-page                                                         /tmp/y.i        4096
1219
put-page                                                         /tmp/t.t        4096
1220
put-page                                                         /tmp/y.i        4096
1221
put-page                                                         /tmp/y.i        4096
1222
put-page                                                         /tmp/y.i        4096
1223
put-page                                                  /tmp/dtrace.out        4096
1224
put-page                                                  /tmp/dtrace.out        4096
1225
put-page                                                         /tmp/y.i        4096
1226
put-page                                                  /tmp/dtrace.out        4096
1227
put-page                                                         /tmp/y.i        4096
1228
put-page                                                         /tmp/y.i        4096
1229
put-page                                                  /tmp/dtrace.out        4096
1230
put-page                                                         /tmp/y.i        4096
1231
put-page                                                         /tmp/y.i        4096
1232
put-page                                                         /tmp/y.i        4096
1233
put-page                                                         /tmp/y.i        4096
1234
put-page                                                         /tmp/y.i        4096
1235
put-page                                /etc/svc/volatile/init-next.state        4096
1236
put-page                                            /var/run/sendmail.pid        4096
1237
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1238
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1239
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1240
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1241
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1242
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1243
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1244
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1245
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1246
put-page                         /etc/svc/volatile/filesystem-autofs.lock        4096
1247
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1248
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1249
put-page                                           /var/run/in.routed.pid        4096
1250
put-page                                             /var/run/in.ndpd.pid        4096
1251
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1252
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1253
put-page                         /etc/svc/volatile/sqlite_2EQn4kUOK9p6ubf        4096
1254
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1255
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1256
put-page                                            /var/run/cups/certs/0        4096
1257
put-page                                                /var/run/sshd.pid        4096
1258
put-page                                    /var/run/ConsoleKit/database~        4096
1259
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1260
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1261
put-page                                          /var/run/ConsoleKit/pid        4096
1262
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1263
put-page                                            /tmp/ogl_select100222        4096
1264
put-page                                                         /tmp/y.i        4096
1265
put-page                                              /var/run/syslog.pid        4096
1266
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1267
put-page                                            /var/run/tzsyncxtGrhg        4096
1268
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1269
put-page                                              /var/run/powerd.pid        4096
1270
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1271
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1272
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1273
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1274
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1275
put-page                                                /var/run/dbus/pid        4096
1276
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1277
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1278
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1279
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1280
put-page                                            /var/run/bootadm.lock        4096
1281
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1282
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1283
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1284
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1285
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1286
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1287
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1288
put-page                         /etc/svc/volatile/ipadm/aobjmap.conf.new        4096
1289
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1290
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1291
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1292
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1293
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1294
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1295
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1296
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1297
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1298
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1299
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1300
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1301
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1302
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1303
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1304
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1305
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1306
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1307
put-page                                        /var/run/rcm_daemon_state        4096
1308
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1309
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1310
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1311
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1312
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1313
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1314
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1315
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1316
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1317
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1318
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1319
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1320
put-page                                                /var/run/hald/pid        4096
1321
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1322
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1323
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1324
put-page       /etc/svc/volatile/dladm/network-datalink-management:default.cache.new        4096
1325
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1326
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1327
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1328
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1329
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1330
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1331
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1332
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1333
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1334
put-page                         /etc/svc/volatile/sqlite_IgUABNLUOMj1EL6        4096
1335
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1336
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1337
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1338
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1339
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1340
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1341
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1342
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1343
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1344
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1345
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1346
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1347
put-page                                                         /tmp/y.i        4096
1348
put-page                                                         /tmp/y.i        4096
1349
put-page                                                         /tmp/y.i        4096
1350
put-page                                                         /tmp/t.t        4096
1351
put-page                                                         /tmp/y.i        4096
1352
put-page                                                         /tmp/t.t        4096
1353
put-page                                                         /tmp/y.i        4096
1354
put-page                                                         /tmp/y.i        4096
1355
put-page                                                         /tmp/y.i        4096
1356
put-page                                                  /tmp/dtrace.out        4096
1357
put-page                                                  /tmp/dtrace.out        4096
1358
put-page                                                         /tmp/y.i        4096
1359
put-page                                                  /tmp/dtrace.out        4096
1360
put-page                                                         /tmp/y.i        4096
1361
put-page                                                         /tmp/y.i        4096
1362
put-page                                                  /tmp/dtrace.out        4096
1363
put-page                                                         /tmp/y.i        4096
1364
put-page                                                         /tmp/y.i        4096
1365
put-page                                                         /tmp/y.i        4096
1366
put-page                                                         /tmp/y.i        4096
1367
put-page                                                         /tmp/y.i        4096
1368
put-page                                /etc/svc/volatile/init-next.state        4096
1369
put-page                                            /var/run/sendmail.pid        4096
1370
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1371
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1372
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1373
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1374
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1375
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1376
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1377
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1378
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1379
put-page                         /etc/svc/volatile/filesystem-autofs.lock        4096
1380
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1381
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1382
put-page                                           /var/run/in.routed.pid        4096
1383
put-page                                             /var/run/in.ndpd.pid        4096
1384
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1385
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1386
put-page                         /etc/svc/volatile/sqlite_2EQn4kUOK9p6ubf        4096
1387
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1388
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1389
put-page                                            /var/run/cups/certs/0        4096
1390
put-page                                                /var/run/sshd.pid        4096
1391
put-page                                    /var/run/ConsoleKit/database~        4096
1392
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1393
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1394
put-page                                          /var/run/ConsoleKit/pid        4096
1395
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1396
put-page                                            /tmp/ogl_select100222        4096
1397
put-page                                                         /tmp/y.i        4096
1398
put-page                                              /var/run/syslog.pid        4096
1399
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1400
put-page                                            /var/run/tzsyncxtGrhg        4096
1401
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1402
put-page                                              /var/run/powerd.pid        4096
1403
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1404
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1405
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1406
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1407
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1408
put-page                                                /var/run/dbus/pid        4096
1409
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1410
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1411
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1412
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1413
put-page                                            /var/run/bootadm.lock        4096
1414
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1415
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1416
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1417
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1418
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1419
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1420
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1421
put-page                         /etc/svc/volatile/ipadm/aobjmap.conf.new        4096
1422
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1423
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1424
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1425
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1426
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1427
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1428
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1429
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1430
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1431
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1432
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1433
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1434
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1435
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1436
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1437
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1438
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1439
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1440
put-page                                        /var/run/rcm_daemon_state        4096
1441
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1442
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1443
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1444
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1445
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1446
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1447
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1448
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1449
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1450
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1451
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1452
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1453
put-page                                                /var/run/hald/pid        4096
1454
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1455
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1456
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1457
put-page       /etc/svc/volatile/dladm/network-datalink-management:default.cache.new        4096
1458
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1459
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1460
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1461
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1462
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1463
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1464
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1465
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1466
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1467
put-page                         /etc/svc/volatile/sqlite_IgUABNLUOMj1EL6        4096
1468
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1469
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1470
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1471
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1472
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1473
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1474
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1475
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1476
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1477
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1478
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1479
put-page                              /etc/svc/volatile/svc_nonpersist.db        4096
1480

    
1481

    
1482

    
1483
# -----------------------------------------------------------------------------
1484
# script 'bitesize.d' below:
1485
#                #!/usr/sbin/dtrace -s
1486
#                /*
1487
#                 * bitesize.d - analyse disk I/O size by process.
1488
#                 *              Written using DTrace (Solaris 10 3/05).
1489
#                 *
1490
#                 * This produces a report for the size of disk events caused by
1491
#                 * processes. These are the disk events sent by the block I/O driver.
1492
#                 *
1493
#                 * If applications must use the disks, we generally prefer they do so
1494
#                 * with large I/O sizes.
1495
#                 *
1496
#                 * $Id: bitesize.d 3 2007-08-01 10:50:08Z brendan $
1497
#                 *
1498
#                 * USAGE:      bitesize.d      # wait several seconds, then hit Ctrl-C
1499
#                 *
1500
#                 * FIELDS:
1501
#                 *             PID             process ID
1502
#                 *             CMD             command and argument list
1503
#                 *             value           size in bytes
1504
#                 *             count           number of I/O operations
1505
#                 *
1506
#                 * NOTES:
1507
#                 *
1508
#                 * The application may be requesting smaller sized operations, which
1509
#                 * are being rounded up to the nearest sector size or UFS block size.
1510
#                 * To analyse what the application is requesting, DTraceToolkit programs
1511
#                 * such as Proc/fddist may help.
1512
#                 *
1513
#                 * SEE ALSO: seeksize.d, iosnoop
1514
#                 *
1515
#                 * COPYRIGHT: Copyright (c) 2006 Brendan Gregg.
1516
#                 *
1517
#                 * CDDL HEADER START
1518
#                 *
1519
#                 *  The contents of this file are subject to the terms of the
1520
#                 *  Common Development and Distribution License, Version 1.0 only
1521
#                 *  (the "License").  You may not use this file except in compliance
1522
#                 *  with the License.
1523
#                 *
1524
#                 *  You can obtain a copy of the license at Docs/cddl1.txt
1525
#                 *  or http://www.opensolaris.org/os/licensing.
1526
#                 *  See the License for the specific language governing permissions
1527
#                 *  and limitations under the License.
1528
#                 *
1529
#                 * CDDL HEADER END
1530
#                 *
1531
#                 * 31-Mar-2004 Brendan Gregg   Created this, build 51.
1532
#                 * 10-Oct-2004    "      "     Rewrote to use the io provider, build 63.
1533
#                 * 18-Feb-2006    "      "     Last update.
1534
#                 */
1535
#
1536
#                #pragma D option quiet
1537
#
1538
#                /*
1539
#                 * Print header
1540
#                 */
1541
#                dtrace:::BEGIN
1542
#                {
1543
#                       printf("Tracing... Hit Ctrl-C to end or wait 60 seconds.\n");
1544
#                }
1545
#
1546
#                /*
1547
#                 * Process io start
1548
#                 */
1549
#                io:::start
1550
#                {
1551
#                       /* fetch details */
1552
#                       this->size = args[0]->b_bcount;
1553
#
1554
#                       /* store details */
1555
#                       @Size[pid, curpsinfo->pr_psargs] = quantize(this->size);
1556
#                }
1557
#
1558
#                /*
1559
#                 * Print final report
1560
#                 */
1561
#                dtrace:::END
1562
#                {
1563
#                       printf("\n%8s  %s\n", "PID", "CMD");
1564
#                       printa("%8d  %S\n%@d\n", @Size);
1565
#                }
1566
#
1567
#                tick-60s
1568
#                {
1569
#                        exit(0);
1570
#                }
1571
#
1572
# -----------------------------------------------------------------------------
1573
# script output/run for ./bitesize.d
1574
# ----------------------------
1575
lab10# ./bitesize.d
1576
Tracing... Hit Ctrl-C to end or wait 60 seconds.
1577

    
1578
     PID  CMD
1579
       6  zpool-rpool\0
1580

    
1581
           value  ------------- Distribution ------------- count
1582
             256 |                                         0
1583
             512 |@@@@@@                                   33
1584
            1024 |@@@@@@@@@                                47
1585
            2048 |@@@@@@@@@@@@@@                           73
1586
            4096 |@@@@@@@@                                 42
1587
            8192 |@@                                       11
1588
           16384 |@                                        4
1589
           32768 |                                         0
1590

    
1591

    
1592

    
1593

    
1594
# -----------------------------------------------------------------------------
1595
# script 'hotspot.d' below:
1596
#                #!/usr/sbin/dtrace -s
1597
#                /*
1598
#                 * hotspot.d - plot disk event by location, look for hotspots.
1599
#                 *             Written in DTrace (Solaris 10 3/05).
1600
#                 *
1601
#                 * This simple DTrace script determines if disk activity is occuring in
1602
#                 * the one place - a "hotspot". This helps us understand the system's usage
1603
#                 * of a disk, it does not imply that the existance or not of a hotspot is
1604
#                 * good or bad (often may be good, less seeking).
1605
#                 *
1606
#                 * $Id: hotspot.d 3 2007-08-01 10:50:08Z brendan $
1607
#                 *
1608
#                 * USAGE:       hotspot.d       # hit Ctrl-C to end
1609
#                 *
1610
#                 * FIELDS:
1611
#                 *              Disk            disk instance name
1612
#                 *              Major           driver major number
1613
#                 *              Minor           driver minor number
1614
#                 *              value           location, by megabyte
1615
#                 *              count           number of I/O operations
1616
#                 *
1617
#                 * COPYRIGHT: Copyright (c) 2005, 2006 Brendan Gregg.
1618
#                 *
1619
#                 * CDDL HEADER START
1620
#                 *
1621
#                 *  The contents of this file are subject to the terms of the
1622
#                 *  Common Development and Distribution License, Version 1.0 only
1623
#                 *  (the "License").  You may not use this file except in compliance
1624
#                 *  with the License.
1625
#                 *
1626
#                 *  You can obtain a copy of the license at Docs/cddl1.txt
1627
#                 *  or http://www.opensolaris.org/os/licensing.
1628
#                 *  See the License for the specific language governing permissions
1629
#                 *  and limitations under the License.
1630
#                 *
1631
#                 * CDDL HEADER END
1632
#                 *
1633
#                 * 07-May-2005 Brendan Gregg   Created this.
1634
#                 * 20-Apr-2006    "      "     Last update.
1635
#                 */
1636
#
1637
#                #pragma D option quiet
1638
#
1639
#                inline int DISK_MB_MAX = 1000000;      /* max size of a single disk */
1640
#                inline int REPORT_SCALE_MB = 1000;     /* output step size for report */
1641
#
1642
#                /*
1643
#                 * Print header
1644
#                 */
1645
#                dtrace:::BEGIN
1646
#                {
1647
#                       printf("Tracing... Hit Ctrl-C to end or wait 60 seconds.\n");
1648
#                }
1649
#
1650
#                /*
1651
#                 * Process disk event
1652
#                 */
1653
#                io:::start
1654
#                {
1655
#                       this->mb = args[0]->b_blkno / 2048;
1656
#                       @Block[args[1]->dev_statname, args[1]->dev_major, args[1]->dev_minor] =
1657
#                           lquantize(this->mb, 0, DISK_MB_MAX, REPORT_SCALE_MB);
1658
#                }
1659
#
1660
#                /*
1661
#                 * Print final report
1662
#                 */
1663
#                dtrace:::END
1664
#                {
1665
#                       printa("Disk: %s   Major,Minor: %d,%d\n%@d\n", @Block);
1666
#                }
1667
#
1668
#                tick-60s
1669
#                {
1670
#                        exit(0);
1671
#                }
1672
#
1673
# -----------------------------------------------------------------------------
1674
# script output/run for ./hotspot.d
1675
# ----------------------------
1676
lab10# ./hotspot.d
1677
Tracing... Hit Ctrl-C to end or wait 60 seconds.
1678
Disk: sd3   Major,Minor: 84,192
1679

    
1680
           value  ------------- Distribution ------------- count
1681
             < 0 |                                         0
1682
               0 |@                                        2
1683
            1000 |                                         0
1684
            2000 |                                         0
1685
            3000 |                                         0
1686
            4000 |                                         0
1687
            5000 |                                         0
1688
            6000 |                                         0
1689
            7000 |                                         0
1690
            8000 |                                         0
1691
            9000 |                                         0
1692
           10000 |                                         0
1693
           11000 |                                         0
1694
           12000 |                                         0
1695
           13000 |                                         0
1696
           14000 |                                         0
1697
           15000 |                                         0
1698
           16000 |                                         0
1699
           17000 |                                         0
1700
           18000 |                                         0
1701
           19000 |                                         0
1702
           20000 |                                         0
1703
           21000 |                                         0
1704
           22000 |                                         0
1705
           23000 |                                         0
1706
           24000 |                                         0
1707
           25000 |                                         0
1708
           26000 |                                         0
1709
           27000 |                                         0
1710
           28000 |                                         0
1711
           29000 |                                         0
1712
           30000 |                                         0
1713
           31000 |                                         0
1714
           32000 |                                         0
1715
           33000 |                                         0
1716
           34000 |                                         0
1717
           35000 |                                         0
1718
           36000 |                                         0
1719
           37000 |                                         0
1720
           38000 |                                         0
1721
           39000 |                                         0
1722
           40000 |                                         0
1723
           41000 |                                         0
1724
           42000 |                                         0
1725
           43000 |                                         0
1726
           44000 |                                         0
1727
           45000 |                                         0
1728
           46000 |                                         0
1729
           47000 |                                         0
1730
           48000 |                                         0
1731
           49000 |                                         0
1732
           50000 |                                         0
1733
           51000 |                                         0
1734
           52000 |                                         0
1735
           53000 |                                         0
1736
           54000 |                                         0
1737
           55000 |                                         0
1738
           56000 |                                         0
1739
           57000 |                                         0
1740
           58000 |                                         0
1741
           59000 |                                         0
1742
           60000 |                                         0
1743
           61000 |                                         0
1744
           62000 |                                         0
1745
           63000 |                                         0
1746
           64000 |                                         0
1747
           65000 |@@@@@@@@@@@@@@@                          24
1748
           66000 |                                         0
1749
           67000 |                                         0
1750
           68000 |                                         0
1751
           69000 |                                         0
1752
           70000 |                                         0
1753
           71000 |                                         0
1754
           72000 |                                         0
1755
           73000 |                                         0
1756
           74000 |                                         0
1757
           75000 |                                         0
1758
           76000 |                                         0
1759
           77000 |                                         0
1760
           78000 |                                         0
1761
           79000 |@@@@@@@                                  11
1762
           80000 |                                         0
1763
           81000 |                                         0
1764
           82000 |                                         0
1765
           83000 |                                         0
1766
           84000 |                                         0
1767
           85000 |                                         0
1768
           86000 |                                         0
1769
           87000 |                                         0
1770
           88000 |                                         0
1771
           89000 |                                         0
1772
           90000 |                                         0
1773
           91000 |                                         0
1774
           92000 |                                         0
1775
           93000 |                                         0
1776
           94000 |                                         0
1777
           95000 |                                         0
1778
           96000 |                                         0
1779
           97000 |                                         0
1780
           98000 |@@@@@@@@@@@@@@@@                         25
1781
           99000 |                                         0
1782
          100000 |                                         0
1783
          101000 |                                         0
1784
          102000 |                                         0
1785
          103000 |                                         0
1786
          104000 |                                         0
1787
          105000 |                                         0
1788
          106000 |                                         0
1789
          107000 |                                         0
1790
          108000 |                                         0
1791
          109000 |                                         0
1792
          110000 |                                         0
1793
          111000 |                                         0
1794
          112000 |                                         0
1795
          113000 |                                         0
1796
          114000 |@                                        2
1797
          115000 |                                         0
1798

    
1799
Disk: sd2   Major,Minor: 84,128
1800

    
1801
           value  ------------- Distribution ------------- count
1802
             < 0 |                                         0
1803
               0 |@                                        2
1804
            1000 |                                         0
1805
            2000 |                                         0
1806
            3000 |                                         0
1807
            4000 |                                         0
1808
            5000 |                                         0
1809
            6000 |                                         0
1810
            7000 |                                         0
1811
            8000 |                                         0
1812
            9000 |                                         0
1813
           10000 |                                         0
1814
           11000 |                                         0
1815
           12000 |                                         0
1816
           13000 |                                         0
1817
           14000 |                                         0
1818
           15000 |                                         0
1819
           16000 |                                         0
1820
           17000 |                                         0
1821
           18000 |                                         0
1822
           19000 |                                         0
1823
           20000 |                                         0
1824
           21000 |                                         0
1825
           22000 |                                         0
1826
           23000 |                                         0
1827
           24000 |                                         0
1828
           25000 |                                         0
1829
           26000 |                                         0
1830
           27000 |                                         0
1831
           28000 |                                         0
1832
           29000 |                                         0
1833
           30000 |                                         0
1834
           31000 |                                         0
1835
           32000 |                                         0
1836
           33000 |                                         0
1837
           34000 |                                         0
1838
           35000 |                                         0
1839
           36000 |                                         0
1840
           37000 |                                         0
1841
           38000 |                                         0
1842
           39000 |                                         0
1843
           40000 |                                         0
1844
           41000 |                                         0
1845
           42000 |                                         0
1846
           43000 |                                         0
1847
           44000 |                                         0
1848
           45000 |                                         0
1849
           46000 |                                         0
1850
           47000 |                                         0
1851
           48000 |                                         0
1852
           49000 |                                         0
1853
           50000 |                                         0
1854
           51000 |                                         0
1855
           52000 |                                         0
1856
           53000 |                                         0
1857
           54000 |                                         0
1858
           55000 |                                         0
1859
           56000 |                                         0
1860
           57000 |                                         0
1861
           58000 |                                         0
1862
           59000 |                                         0
1863
           60000 |                                         0
1864
           61000 |                                         0
1865
           62000 |                                         0
1866
           63000 |                                         0
1867
           64000 |                                         0
1868
           65000 |@@@@@@@@@@@@@@@                          25
1869
           66000 |                                         0
1870
           67000 |                                         0
1871
           68000 |                                         0
1872
           69000 |                                         0
1873
           70000 |                                         0
1874
           71000 |                                         0
1875
           72000 |                                         0
1876
           73000 |                                         0
1877
           74000 |                                         0
1878
           75000 |                                         0
1879
           76000 |                                         0
1880
           77000 |                                         0
1881
           78000 |                                         0
1882
           79000 |@@@@@@@                                  11
1883
           80000 |                                         0
1884
           81000 |                                         0
1885
           82000 |                                         0
1886
           83000 |                                         0
1887
           84000 |                                         0
1888
           85000 |                                         0
1889
           86000 |                                         0
1890
           87000 |                                         0
1891
           88000 |                                         0
1892
           89000 |                                         0
1893
           90000 |                                         0
1894
           91000 |                                         0
1895
           92000 |                                         0
1896
           93000 |                                         0
1897
           94000 |                                         0
1898
           95000 |                                         0
1899
           96000 |                                         0
1900
           97000 |                                         0
1901
           98000 |@@@@@@@@@@@@@@@                          25
1902
           99000 |                                         0
1903
          100000 |                                         0
1904
          101000 |                                         0
1905
          102000 |                                         0
1906
          103000 |                                         0
1907
          104000 |                                         0
1908
          105000 |                                         0
1909
          106000 |                                         0
1910
          107000 |                                         0
1911
          108000 |                                         0
1912
          109000 |                                         0
1913
          110000 |                                         0
1914
          111000 |                                         0
1915
          112000 |                                         0
1916
          113000 |                                         0
1917
          114000 |@                                        2
1918
          115000 |                                         0
1919

    
1920

    
1921

    
1922

    
1923
# -----------------------------------------------------------------------------
1924
# script 'iofile.d' below:
1925
#                #!/usr/sbin/dtrace -s
1926
#                /*
1927
#                 * iofile.d - I/O wait time by filename and process.
1928
#                 *            Written using DTrace (Solaris 10 3/05).
1929
#                 *
1930
#                 * This prints the total I/O wait times for each filename by process.
1931
#                 * This can help determine why an application is performing poorly by
1932
#                 * identifying which file they are waiting on, and the total times.
1933
#                 * Both disk and NFS I/O are measured.
1934
#                 *
1935
#                 * $Id: iofile.d 3 2007-08-01 10:50:08Z brendan $
1936
#                 *
1937
#                 * USAGE:      iofile.d        # wait, then hit Ctrl-C to end
1938
#                 *
1939
#                 * FIELDS:
1940
#                 *             PID             Process ID
1941
#                 *             CMD             Process name
1942
#                 *             TIME            Total wait time for disk events, us
1943
#                 *             FILE            File pathname
1944
#                 *
1945
#                 * BASED ON: /usr/demo/dtrace/iocpu.d
1946
#                 *
1947
#                 * SEE ALSO: iosnoop, iotop
1948
#                 *
1949
#                 * PORTIONS: Copyright (c) 2005, 2006 Brendan Gregg.
1950
#                 *
1951
#                 * CDDL HEADER START
1952
#                 *
1953
#                 *  The contents of this file are subject to the terms of the
1954
#                 *  Common Development and Distribution License, Version 1.0 only
1955
#                 *  (the "License").  You may not use this file except in compliance
1956
#                 *  with the License.
1957
#                 *
1958
#                 *  You can obtain a copy of the license at Docs/cddl1.txt
1959
#                 *  or http://www.opensolaris.org/os/licensing.
1960
#                 *  See the License for the specific language governing permissions
1961
#                 *  and limitations under the License.
1962
#                 *
1963
#                 * CDDL HEADER END
1964
#                 *
1965
#                 * 24-Jul-2005 Brendan Gregg   Created this.
1966
#                 * 20-Apr-2006    "      "     Last update.
1967
#                 */
1968
#
1969
#                #pragma D option quiet
1970
#
1971
#                /* print header */
1972
#                dtrace:::BEGIN
1973
#                {
1974
#                       printf("Tracing... Hit Ctrl-C to end or wait 60 seconds.\n");
1975
#                }
1976
#
1977
#                /* save time at start */
1978
#                io:::wait-start
1979
#                {
1980
#                       self->start = timestamp;
1981
#                }
1982
#
1983
#                /* process event */
1984
#                io:::wait-done
1985
#                /self->start/
1986
#                {
1987
#                       /*
1988
#                        * wait-done is used as we are measing wait times. It also
1989
#                        * is triggered when the correct thread is on the CPU, obviating
1990
#                        * the need to link process details to the start event.
1991
#                        */
1992
#                       this->elapsed = timestamp - self->start;
1993
#                       @files[pid, execname, args[2]->fi_pathname] = sum(this->elapsed);
1994
#                       self->start = 0;
1995
#                }
1996
#
1997
#                /* print report */
1998
#                dtrace:::END
1999
#                {
2000
#                       normalize(@files, 1000);
2001
#                       printf("%6s %-12s %8s %s\n", "PID", "CMD", "TIME", "FILE");
2002
#                       printa("%6d %-12.12s %@8d %s\n", @files);
2003
#                }
2004
#
2005
#                tick-60s
2006
#                {
2007
#                        exit(0);
2008
#                }
2009
#
2010
# -----------------------------------------------------------------------------
2011
# script output/run for ./iofile.d
2012
# ----------------------------
2013
lab10# ./iofile.d
2014
Tracing... Hit Ctrl-C to end or wait 60 seconds.
2015
   PID CMD              TIME FILE
2016

    
2017

    
2018

    
2019
# -----------------------------------------------------------------------------
2020
# script 'iofileb.d' below:
2021
#                #!/usr/sbin/dtrace -s
2022
#                /*
2023
#                 * iofileb.d - I/O bytes by filename and process.
2024
#                 *             Written using DTrace (Solaris 10 3/05).
2025
#                 *
2026
#                 * This prints a summary of requested disk activity by pathname,
2027
#                 * providing totals of the I/O events in bytes. It is a companion to the
2028
#                 * iofile.d script - which prints in terms of I/O wait time, not bytes.
2029
#                 * I/O wait time is a better metric for understanding performance issues.
2030
#                 * Both disk and NFS I/O are measured.
2031
#                 *
2032
#                 * $Id: iofileb.d 3 2007-08-01 10:50:08Z brendan $
2033
#                 *
2034
#                 * USAGE:      iofileb.d       # wait several seconds, then hit Ctrl-C
2035
#                 *
2036
#                 * FIELDS:
2037
#                 *             PID     process ID
2038
#                 *             CMD     command name
2039
#                 *             KB      Kilobytes of disk I/O
2040
#                 *             FILE    Full pathname of the file
2041
#                 *
2042
#                 * COPYRIGHT: Copyright (c) 2006 Brendan Gregg.
2043
#                 *
2044
#                 * CDDL HEADER START
2045
#                 *
2046
#                 *  The contents of this file are subject to the terms of the
2047
#                 *  Common Development and Distribution License, Version 1.0 only
2048
#                 *  (the "License").  You may not use this file except in compliance
2049
#                 *  with the License.
2050
#                 *
2051
#                 *  You can obtain a copy of the license at Docs/cddl1.txt
2052
#                 *  or http://www.opensolaris.org/os/licensing.
2053
#                 *  See the License for the specific language governing permissions
2054
#                 *  and limitations under the License.
2055
#                 *
2056
#                 * CDDL HEADER END
2057
#                 *
2058
#                 * 20-Feb-2006 Brendan Gregg   Created this.
2059
#                 * 20-Feb-2006    "      "     Last update.
2060
#                 */
2061
#
2062
#                #pragma D option quiet
2063
#
2064
#                dtrace:::BEGIN
2065
#                {
2066
#                       printf("Tracing... Hit Ctrl-C to end or wait 60 seconds.\n");
2067
#                }
2068
#
2069
#                io:::start
2070
#                {
2071
#                       @files[pid, execname, args[2]->fi_pathname] = sum(args[0]->b_bcount);
2072
#                }
2073
#
2074
#                dtrace:::END
2075
#                {
2076
#                       normalize(@files, 1024);
2077
#                       printf("%6s %-12s %6s %s\n", "PID", "CMD", "KB", "FILE");
2078
#                       printa("%6d %-12.12s %@6d %s\n", @files);
2079
#                }
2080
#
2081
#                tick-60s
2082
#                {
2083
#                        exit(0);
2084
#                }
2085
#
2086
# -----------------------------------------------------------------------------
2087
# script output/run for ./iofileb.d
2088
# ----------------------------
2089
lab10# ./iofileb.d
2090
Tracing... Hit Ctrl-C to end or wait 60 seconds.
2091
   PID CMD              KB FILE
2092
     6 zpool-rpool     488 <none>
2093

    
2094

    
2095

    
2096
# -----------------------------------------------------------------------------
2097
# script 'seeksize.d' below:
2098
#                #!/usr/sbin/dtrace -s
2099
#                /*
2100
#                 * seeksize.d - analyse disk head seek distance by process.
2101
#                 *              Written using DTrace (Solaris 10 3/05).
2102
#                 *
2103
#                 * Disk I/O events caused by processes will in turn cause the disk heads
2104
#                 * to seek. This program analyses those seeks, so that we can determine
2105
#                 * if processes are causing the disks to seek in a "random" or "sequential"
2106
#                 * manner.
2107
#                 *
2108
#                 * $Id: seeksize.d 3 2007-08-01 10:50:08Z brendan $
2109
#                 *
2110
#                 * USAGE:      seeksize.d              # wait several seconds, then hit Ctrl-C
2111
#                 *
2112
#                 * FIELDS:
2113
#                 *             PID     process ID
2114
#                 *             CMD     command and argument list
2115
#                 *             value   distance in disk blocks (sectors)
2116
#                 *             count   number of I/O operations
2117
#                 *
2118
#                 * SEE ALSO: bitesize.d, iosnoop
2119
#                 *
2120
#                 * COPYRIGHT: Copyright (c) 2006 Brendan Gregg.
2121
#                 *
2122
#                 * CDDL HEADER START
2123
#                 *
2124
#                 *  The contents of this file are subject to the terms of the
2125
#                 *  Common Development and Distribution License, Version 1.0 only
2126
#                 *  (the "License").  You may not use this file except in compliance
2127
#                 *  with the License.
2128
#                 *
2129
#                 *  You can obtain a copy of the license at Docs/cddl1.txt
2130
#                 *  or http://www.opensolaris.org/os/licensing.
2131
#                 *  See the License for the specific language governing permissions
2132
#                 *  and limitations under the License.
2133
#                 *
2134
#                 * CDDL HEADER END
2135
#                 *
2136
#                 * 11-Sep-2004 Brendan Gregg   Created this.
2137
#                 * 10-Oct-2004    "      "     Rewrote to use the io provider.
2138
#                 * 20-Apr-2006    "      "     Last update.
2139
#                 */
2140
#
2141
#                #pragma D option quiet
2142
#
2143
#                /*
2144
#                 * Print header
2145
#                 */
2146
#                dtrace:::BEGIN
2147
#                {
2148
#                       printf("Tracing... Hit Ctrl-C to end or wait 60 seconds.\n");
2149
#                }
2150
#
2151
#                self int last[dev_t];
2152
#
2153
#                /*
2154
#                 * Process io start
2155
#                 */
2156
#                io:genunix::start
2157
#                /self->last[args[0]->b_edev] != 0/
2158
#                {
2159
#                       /* calculate seek distance */
2160
#                       this->last = self->last[args[0]->b_edev];
2161
#                       this->dist = (int)(args[0]->b_blkno - this->last) > 0 ?
2162
#                           args[0]->b_blkno - this->last : this->last - args[0]->b_blkno;
2163
#
2164
#                       /* store details */
2165
#                       @Size[pid, curpsinfo->pr_psargs] = quantize(this->dist);
2166
#                }
2167
#
2168
#                io:genunix::start
2169
#                {
2170
#                       /* save last position of disk head */
2171
#                       self->last[args[0]->b_edev] = args[0]->b_blkno +
2172
#                           args[0]->b_bcount / 512;
2173
#                }
2174
#
2175
#                /*
2176
#                 * Print final report
2177
#                 */
2178
#                dtrace:::END
2179
#                {
2180
#                       printf("\n%8s  %s\n", "PID", "CMD");
2181
#                       printa("%8d  %S\n%@d\n", @Size);
2182
#                }
2183
#
2184
#                tick-60s
2185
#                {
2186
#                        exit(0);
2187
#                }
2188
#
2189
# -----------------------------------------------------------------------------
2190
# script output/run for ./seeksize.d
2191
# ----------------------------
2192
lab10# ./seeksize.d
2193
Tracing... Hit Ctrl-C to end or wait 60 seconds.
2194

    
2195
     PID  CMD
2196

    
2197

    
2198

    
2199
# -----------------------------------------------------------------------------
2200
# script 'iopending' below:
2201
#                #!/usr/bin/ksh
2202
#                #
2203
#                # iopending - Print a plot for the number of pending disk I/O events.
2204
#                #             Written using DTrace (Solaris 10 3/05).
2205
#                #
2206
#                # This is measuring disk events that have made it past system caches.
2207
#                # By plotting a distribution graph of the number of pending events, the
2208
#                # "serialness" or "parallelness" of disk behaviour can be distinguished.
2209
#                #
2210
#                # $Id: iopending 3 2007-08-01 10:50:08Z brendan $
2211
#                #
2212
#                # USAGE:       iopending [-c] [-d device] [-f filename]
2213
#                #                        [-m mount_point] [interval [count]]
2214
#                #
2215
#                #              -c              # clear the screen
2216
#                #              -d device       # instance name to snoop (eg, dad0)
2217
#                #              -f filename     # full pathname of file to snoop
2218
#                #              -m mount_point  # this FS only (will skip raw events)
2219
#                #      eg,
2220
#                #              iopending       # default output, 5 second intervals
2221
#                #              iopending 1     # 1 second samples
2222
#                #              iopending -c    # clear the screen
2223
#                #              iopending 5 12  # print 12 x 5 second samples
2224
#                #
2225
#                # FIELDS:
2226
#                #              value           number of pending events, 0 == idle
2227
#                #              count           number of samples @ 1000 Hz
2228
#                #              load            1 min load average
2229
#                #              disk_r          total disk read Kbytes for sample
2230
#                #              disk_w          total disk write Kbytes for sample
2231
#                #
2232
#                # SEE ALSO: iosnoop, iotop
2233
#                #
2234
#                # IDEA: Dr Rex di Bona (Sydney, Australia)
2235
#                #
2236
#                # COPYRIGHT: Copyright (c) 2005, 2006 Brendan Gregg.
2237
#                #
2238
#                # CDDL HEADER START
2239
#                #
2240
#                #  The contents of this file are subject to the terms of the
2241
#                #  Common Development and Distribution License, Version 1.0 only
2242
#                #  (the "License").  You may not use this file except in compliance
2243
#                #  with the License.
2244
#                #
2245
#                #  You can obtain a copy of the license at Docs/cddl1.txt
2246
#                #  or http://www.opensolaris.org/os/licensing.
2247
#                #  See the License for the specific language governing permissions
2248
#                #  and limitations under the License.
2249
#                #
2250
#                # CDDL HEADER END
2251
#                #
2252
#                # Author: Brendan Gregg  [Sydney, Australia]
2253
#                #
2254
#                # 01-Nov-2005  Brendan Gregg   Created this.
2255
#                # 20-Apr-2006     "      "     Last update.
2256
#                #
2257
#
2258
#
2259
#                ##############################
2260
#                # --- Process Arguments ---
2261
#                #
2262
#
2263
#                ### default variables
2264
#                opt_device=0; opt_file=0; opt_mount=0; opt_clear=0;
2265
#                opt_def=1; filter=0; device=.; filename=.; mount=.
2266
#                interval=5; count=-1
2267
#
2268
#                ### process options
2269
#                while getopts cd:f:hm: name
2270
#                do
2271
#                       case $name in
2272
#                       c)      opt_clear=1 ;;
2273
#                       d)      opt_device=1; device=$OPTARG ;;
2274
#                       f)      opt_file=1; filename=$OPTARG ;;
2275
#                       m)      opt_mount=1; mount=$OPTARG ;;
2276
#                       h|?)    cat <<-END >&2
2277
#                               USAGE: iopending [-c] [-d device] [-f filename]
2278
#                                                [-m mount_point] [interval [count]]
2279
#
2280
#                                               -c              # clear the screen
2281
#                                               -d device       # instance name to snoop
2282
#                                               -f filename     # snoop this file only
2283
#                                               -m mount_point  # this FS only
2284
#                                  eg,
2285
#                                       iopending         # default output, 5 second samples
2286
#                                       iopending 1       # 1 second samples
2287
#                                       iopending -m /    # snoop events on filesystem / only
2288
#                                       iopending 5 12    # print 12 x 5 second samples
2289
#                               END
2290
#                               exit 1
2291
#                       esac
2292
#                done
2293
#
2294
#                shift $(( $OPTIND - 1 ))
2295
#
2296
#                ### option logic
2297
#                if [[ "$1" > 0 ]]; then
2298
#                        interval=$1; shift
2299
#                fi
2300
#                if [[ "$1" > 0 ]]; then
2301
#                        count=$1; shift
2302
#                fi
2303
#                if (( opt_device || opt_mount || opt_file )); then
2304
#                       filter=1
2305
#                fi
2306
#                if (( opt_clear )); then
2307
#                        clearstr=`clear`
2308
#                else
2309
#                        clearstr=.
2310
#                fi
2311
#
2312
#
2313
#
2314
#                #################################
2315
#                # --- Main Program, DTrace ---
2316
#                #
2317
#                /usr/sbin/dtrace -n '
2318
#                 /*
2319
#                  * Command line arguments
2320
#                  */
2321
#                 inline int OPT_def    = '$opt_def';
2322
#                 inline int OPT_clear  = '$opt_clear';
2323
#                 inline int OPT_device         = '$opt_device';
2324
#                 inline int OPT_mount  = '$opt_mount';
2325
#                 inline int OPT_file   = '$opt_file';
2326
#                 inline int INTERVAL   = '$interval';
2327
#                 inline int COUNTER    = '$count';
2328
#                 inline int FILTER     = '$filter';
2329
#                 inline string DEVICE  = "'$device'";
2330
#                 inline string FILENAME = "'$filename'";
2331
#                 inline string MOUNT   = "'$mount'";
2332
#                 inline string CLEAR   = "'$clearstr'";
2333
#
2334
#                 inline int MAX_PENDING = 32;  /* max pending value */
2335
#
2336
#                 #pragma D option quiet
2337
#
2338
#                 /*
2339
#                  * Print header
2340
#                  */
2341
#                 dtrace:::BEGIN
2342
#                 {
2343
#                        /* starting values */
2344
#                        counts = COUNTER;
2345
#                        secs = INTERVAL;
2346
#                        disk_r = 0;
2347
#                        disk_w = 0;
2348
#                        pending = 0;
2349
#
2350
#                        printf("Tracing... Please wait.\n");
2351
#                 }
2352
#
2353
#                 /*
2354
#                  * Check event is being traced
2355
#                  */
2356
#                 io:genunix::start,
2357
#                 io:genunix::done
2358
#                 {
2359
#                       /* default is to trace unless filtering, */
2360
#                       this->ok = FILTER ? 0 : 1;
2361
#
2362
#                       /* check each filter, */
2363
#                       (OPT_device == 1 && DEVICE == args[1]->dev_statname)? this->ok = 1 : 1;
2364
#                       (OPT_file == 1 && FILENAME == args[2]->fi_pathname) ? this->ok = 1 : 1;
2365
#                       (OPT_mount == 1 && MOUNT == args[2]->fi_mount)  ? this->ok = 1 : 1;
2366
#                 }
2367
#
2368
#                 /*
2369
#                  * Store entry details
2370
#                  */
2371
#                 io:genunix::start
2372
#                 /this->ok/
2373
#                 {
2374
#                       /* track bytes */
2375
#                       disk_r += args[0]->b_flags & B_READ ? args[0]->b_bcount : 0;
2376
#                       disk_w += args[0]->b_flags & B_READ ? 0 : args[0]->b_bcount;
2377
#
2378
#                       /* increase event pending count */
2379
#                       pending++;
2380
#                 }
2381
#
2382
#                 /*
2383
#                  * Process and Print completion
2384
#                  */
2385
#                 io:genunix::done
2386
#                 /this->ok/
2387
#                 {
2388
#                       /* decrease event pending count */
2389
#                       pending--;
2390
#                 }
2391
#
2392
#                 /*
2393
#                  * Prevent pending from underflowing
2394
#                  * this can happen if this program is started during disk events.
2395
#                  */
2396
#                 io:genunix::done
2397
#                 /pending < 0/
2398
#                 {
2399
#                       pending = 0;
2400
#                 }
2401
#
2402
#                 /*
2403
#                  * Timer
2404
#                  */
2405
#                 profile:::tick-1sec
2406
#                 {
2407
#                       secs--;
2408
#                 }
2409
#
2410
#                 profile:::profile-1000hz
2411
#                 {
2412
#                       @out = lquantize(pending, 0, MAX_PENDING, 1);
2413
#                 }
2414
#
2415
#                 /*
2416
#                  * Print Report
2417
#                  */
2418
#                 profile:::tick-1sec
2419
#                 /secs == 0/
2420
#                 {
2421
#                       /* fetch 1 min load average */
2422
#                       this->load1a  = `hp_avenrun[0] / 65536;
2423
#                       this->load1b  = ((`hp_avenrun[0] % 65536) * 100) / 65536;
2424
#
2425
#                       /* convert counters to Kbytes */
2426
#                       disk_r /= 1024;
2427
#                       disk_w /= 1024;
2428
#
2429
#                       /* print status */
2430
#                       OPT_clear ? printf("%s", CLEAR) : 1;
2431
#                       printf("%Y,  load: %d.%02d,  disk_r: %6d KB,  disk_w: %6d KB",
2432
#                           walltimestamp, this->load1a, this->load1b, disk_r, disk_w);
2433
#
2434
#                       /* print output */
2435
#                       printa(@out);
2436
#
2437
#                       /* clear data */
2438
#                       trunc(@out);
2439
#                       disk_r = 0;
2440
#                       disk_w = 0;
2441
#                       secs = INTERVAL;
2442
#                       counts--;
2443
#                 }
2444
#
2445
#                 /*
2446
#                  * End of program
2447
#                  */
2448
#                 profile:::tick-1sec
2449
#                 /counts == 0/
2450
#                 {
2451
#                       exit(0);
2452
#                 }
2453
#
2454
#                 /*
2455
#                  * Cleanup for Ctrl-C
2456
#                  */
2457
#                 dtrace:::END
2458
#                 {
2459
#                       trunc(@out);
2460
#                 }
2461
#
2462
#                 tick-60s
2463
#                 {
2464
#                        exit(0);
2465
#                 }
2466
#                '
2467
# -----------------------------------------------------------------------------
2468
# script output/run for ./iopending
2469
# ----------------------------
2470
lab10# ./iopending
2471
Tracing... Please wait.
2472
2011 Nov 21 10:22:45,  load: 0.00,  disk_r:      0 KB,  disk_w:    474 KB
2473

    
2474
           value  ------------- Distribution ------------- count
2475
             < 0 |                                         0
2476
               0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@      17409
2477
               1 |                                         5
2478
               2 |@@@@@                                    2539
2479
               3 |                                         2
2480
               4 |                                         1
2481
               5 |                                         0
2482
               6 |                                         2
2483
               7 |                                         1
2484
               8 |                                         6
2485
               9 |                                         0
2486
              10 |                                         35
2487
              11 |                                         0
2488
              12 |                                         0
2489
              13 |                                         0
2490
              14 |                                         0
2491
              15 |                                         0
2492
              16 |                                         0
2493
              17 |                                         0
2494
              18 |                                         1
2495
              19 |                                         3
2496
              20 |                                         0
2497

    
2498
2011 Nov 21 10:22:50,  load: 0.01,  disk_r:      0 KB,  disk_w:      0 KB
2499

    
2500
           value  ------------- Distribution ------------- count
2501
               1 |                                         0
2502
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2503
               3 |                                         0
2504

    
2505
2011 Nov 21 10:22:55,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2506

    
2507
           value  ------------- Distribution ------------- count
2508
               1 |                                         0
2509
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2510
               3 |                                         0
2511

    
2512
2011 Nov 21 10:23:00,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2513

    
2514
           value  ------------- Distribution ------------- count
2515
               1 |                                         0
2516
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2517
               3 |                                         0
2518

    
2519
2011 Nov 21 10:23:05,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2520

    
2521
           value  ------------- Distribution ------------- count
2522
               1 |                                         0
2523
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2524
               3 |                                         0
2525

    
2526
2011 Nov 21 10:23:10,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2527

    
2528
           value  ------------- Distribution ------------- count
2529
               1 |                                         0
2530
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2531
               3 |                                         0
2532

    
2533
2011 Nov 21 10:23:15,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2534

    
2535
           value  ------------- Distribution ------------- count
2536
               1 |                                         0
2537
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2538
               3 |                                         0
2539

    
2540
2011 Nov 21 10:23:20,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2541

    
2542
           value  ------------- Distribution ------------- count
2543
               1 |                                         0
2544
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2545
               3 |                                         0
2546

    
2547
2011 Nov 21 10:23:25,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2548

    
2549
           value  ------------- Distribution ------------- count
2550
               1 |                                         0
2551
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2552
               3 |                                         0
2553

    
2554
2011 Nov 21 10:23:30,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2555

    
2556
           value  ------------- Distribution ------------- count
2557
               1 |                                         0
2558
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2559
               3 |                                         0
2560

    
2561
2011 Nov 21 10:23:35,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2562

    
2563
           value  ------------- Distribution ------------- count
2564
               1 |                                         0
2565
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 20000
2566
               3 |                                         0
2567

    
2568
2011 Nov 21 10:23:40,  load: 0.00,  disk_r:      0 KB,  disk_w:      0 KB
2569

    
2570
           value  ------------- Distribution ------------- count
2571
               1 |                                         0
2572
               2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 19948
2573
               3 |                                         0
2574

    
2575

    
2576

    
2577
# -----------------------------------------------------------------------------
2578
# script output/run for ./hotkernel_60 (based on dtrace tool kit) 
2579
# ----------------------------
2580

    
2581
lab10# ./hotkernel_60
2582
Sampling... Hit Ctrl-C to end. -OR- wait 60 seconds for autoquit
2583

    
2584
FUNCTION                                                COUNT   PCNT
2585
genunix`cyclic_reprogram                                    1   0.0%
2586
genunix`thread_free_prevent                                 1   0.0%
2587
unix`0xfffffffffb86                                         1   0.0%
2588
unix`rw_enter                                               1   0.0%
2589
sd`sdintr                                                   1   0.0%
2590
genunix`scalehrtime                                         1   0.0%
2591
zfs`refcount_is_zero                                        1   0.0%
2592
genunix`sleepq_unlink                                       1   0.0%
2593
genunix`cv_block                                            1   0.0%
2594
apix`apic_unset_idlecpu                                     1   0.0%
2595
unix`tsc_scalehrtime                                        1   0.0%
2596
genunix`disp_lock_exit                                      1   0.0%
2597
genunix`list_destroy                                        1   0.0%
2598
genunix`callout_heap_delete                                 1   0.0%
2599
unix`pg_ev_thread_swtch                                     1   0.0%
2600
unix`clear_int_flag                                         1   0.0%
2601
unix`hati_demap_func                                        1   0.0%
2602
genunix`cv_timedwait_hires                                  1   0.0%
2603
unix`strcmp                                                 1   0.0%
2604
unix`cmt_ev_thread_swtch_pwr                                1   0.0%
2605
genunix`mod_hold_next_by_id                                 1   0.0%
2606
unix`mutex_delay_default                                    1   0.0%
2607
unix`cpu_idle_mwait                                         1   0.0%
2608
genunix`lbolt_event_driven                                  1   0.0%
2609
unix`htable_e2va                                            1   0.0%
2610
unix`cpupm_utilization_event                                1   0.0%
2611
genunix`cv_broadcast                                        1   0.0%
2612
unix`_resume_from_idle                                      1   0.0%
2613
unix`cstate_wakeup                                          1   0.0%
2614
genunix`vmem_seg_create                                     1   0.0%
2615
unix`mul32                                                  1   0.0%
2616
genunix`ddi_dma_sync                                        1   0.0%
2617
unix`hat_memload                                            1   0.0%
2618
unix`atomic_cas_32                                          1   0.0%
2619
zfs`dbuf_verify                                             1   0.0%
2620
unix`rw_exit                                                1   0.0%
2621
genunix`devi_unconfig_common                                1   0.0%
2622
genunix`memcmp                                              1   0.0%
2623
unix`ddi_get32                                              1   0.0%
2624
unix`mutex_owned                                            1   0.0%
2625
genunix`kmem_cache_free_debug                               1   0.0%
2626
unix`cpu_resched                                            1   0.0%
2627
unix`chk_lpg                                                1   0.0%
2628
genunix`gethrtime_unscaled                                  1   0.0%
2629
unix`bitset_in_set                                          1   0.0%
2630
unix`mutex_destroy                                          1   0.0%
2631
genunix`sleepq_insert                                       1   0.0%
2632
genunix`avl_walk                                            1   0.0%
2633
unix`av_dispatch_softvect                                   1   0.0%
2634
unix`idle_exit                                              1   0.0%
2635
unix`lock_clear_splx                                        1   0.0%
2636
unix`av_clear_softint_pending                               1   0.0%
2637
mac_ether`mac_ether_header_info                             1   0.0%
2638
unix`mutex_exit                                             1   0.0%
2639
genunix`cv_unsleep                                          1   0.0%
2640
genunix`kmem_cache_alloc_debug                              1   0.0%
2641
unix`vsnprintf                                              1   0.0%
2642
genunix`timeout                                             1   0.0%
2643
unix`atomic_add_32_nv                                       1   0.0%
2644
apix`apix_dispatch_softint                                  2   0.0%
2645
unix`disp_lowpri_cpu                                        2   0.0%
2646
unix`cmt_ev_thread_swtch                                    2   0.0%
2647
genunix`drv_usectohz                                        2   0.0%
2648
unix`tsc_gethrtimeunscaled                                  2   0.0%
2649
unix`fakesoftint_return                                     2   0.0%
2650
unix`cpu_idle_enter                                         2   0.0%
2651
unix`membar_enter                                           2   0.0%
2652
unix`disp_getwork                                           3   0.0%
2653
dtrace`dtrace_dynvar_clean                                  3   0.0%
2654
genunix`cyclic_softint                                      4   0.0%
2655
genunix`cyclic_coverage                                     4   0.0%
2656
genunix`clock                                               4   0.0%
2657
sha1`sha1_block_data_order                                  5   0.0%
2658
unix`atomic_add_32                                          5   0.0%
2659
unix`lock_try                                               5   0.0%
2660
unix`i86_monitor                                            5   0.0%
2661
genunix`cyclic_coverage_hash                                6   0.0%
2662
genunix`vmem_hash_rescale                                   6   0.0%
2663
genunix`thread_lock                                         7   0.0%
2664
unix`mutex_enter                                            8   0.0%
2665
unix`splr                                                   8   0.0%
2666
unix`do_splx                                                9   0.0%
2667
unix`tsc_read                                              10   0.0%
2668
TS`ts_update_list                                          11   0.0%
2669
genunix`dnlc_purge_vfsp                                    19   0.0%
2670
genunix`fsflush_do_pages                                   28   0.0%
2671
unix`page_nextn                                            31   0.0%
2672
unix`acpi_cpu_cstate                                      414   0.2%
2673
unix`i86_mwait                                         243513  99.7%
2674
lab10#
2675

    
2676

    
2677

    
2678
# -----------------------------------------------------------------------------
2679
# script 'cpu.d' below:
2680
#                #!/usr/sbin/dtrace -s
2681
#
2682
#                syscall:::entry
2683
#                {
2684
#                        self->vstart = vtimestamp;
2685
#                }
2686
#
2687
#                syscall:::return
2688
#                /self->vstart/
2689
#                {
2690
#                        this->oncpu = vtimestamp - self->vstart;
2691
#                        @[probefunc] = avg(this->oncpu);
2692
#                        self->vstart = 0;
2693
#                }
2694
#
2695
#                tick-60s
2696
#                {
2697
#                        exit(0);
2698
#                }
2699
#
2700
# -----------------------------------------------------------------------------
2701
# script output/run for ./cpu.d
2702
# ----------------------------
2703
lab10# ./cpu.d
2704
dtrace: script './cpu.d' matched 465 probes
2705
CPU     ID                    FUNCTION:NAME
2706
  2  65567                        :tick-60s
2707

    
2708
  p_online                                                         82
2709
  sysconfig                                                       222
2710
  getpid                                                          225
2711
  lwp_self                                                        231
2712
  getgid                                                          253
2713
  lwp_sigmask                                                     321
2714
  gtime                                                           346
2715
  getuid                                                          348
2716
  pset                                                            509
2717
  fcntl                                                           526
2718
  sigaction                                                       667
2719
  uucopy                                                          769
2720
  fstat64                                                         812
2721
  clock_gettime                                                   841
2722
  lwp_continue                                                    914
2723
  setitimer                                                      1002
2724
  ioctl                                                          1126
2725
  lwp_kill                                                       1274
2726
  setcontext                                                     1852
2727
  brk                                                            2202
2728
  setuid                                                         2303
2729
  mmap                                                           3064
2730
  close                                                          3314
2731
  setgroups                                                      3350
2732
  stat64                                                         3466
2733
  openat                                                         3496
2734
  read                                                           5135
2735
  waitsys                                                        5543
2736
  doorfs                                                         7100
2737
  pause                                                          7176
2738
  schedctl                                                       8909
2739
  statvfs64                                                      9249
2740
  nanosleep                                                     10093
2741
  lwp_park                                                      11122
2742
  pread                                                         11772
2743
  stat                                                          13443
2744
  writev                                                        14900
2745
  getdents64                                                    16320
2746
  pollsys                                                       20063
2747
  open64                                                        25169
2748
  lwp_create                                                    27477
2749
  open                                                          38048
2750
  forksys                                                      421456
2751
  umount2                                                    19804321
2752
lab10#
2753

    
2754

    
2755

    
2756
# -----------------------------------------------------------------------------
2757
# script output/run a one liner ZFS entries (6 seconds)
2758
# ----------------------------
2759

    
2760

    
2761
lab10#  dtrace -n 'fbt:zfs::entry { @[probefunc] = count(); }  tick-6s { exit(0); }'
2762
dtrace: description 'fbt:zfs::entry ' matched 2335 probes
2763
CPU     ID                    FUNCTION:NAME
2764
  0  65561                         :tick-6s
2765

    
2766
  bpobj_iterate                                                     1
2767
  bpobj_iterate_impl                                                1
2768
  dbuf_free_range                                                   1
2769
  dbuf_undirty                                                      1
2770
  dmu_free_range                                                    1
2771
  dmu_objset_userquota_find_data                                    1
2772
  dmu_zfetch_stream_insert                                          1
2773
  dmu_zfetch_stream_remove                                          1
2774
  dnode_free_range                                                  1
2775
  dnode_sync_free_range                                             1
2776
  free_blocks                                                       1
2777
  sa_hdrsize                                                        1
2778
  txg_list_head                                                     1
2779
  txg_list_next                                                     1
2780
  txg_sync_waiting                                                  1
2781
  vdev_uberblock_sync_list                                          1
2782
  zfs_space_delta_cb                                                1
2783
  zil_clean                                                         1
2784
  zil_itxg_clean                                                    1
2785
  ddt_get_dedup_dspace                                              2
2786
  ddt_get_dedup_histogram                                           2
2787
  ddt_get_dedup_stats                                               2
2788
  ddt_histogram_stat                                                2
2789
  do_userquota_update                                               2
2790
  dsl_dataset_block_freeable                                        2
2791
  dsl_dataset_get_spa                                               2
2792
  dsl_dataset_prev_snap_txg                                         2
2793
  dsl_dataset_sync                                                  2
2794
  dsl_pool_sync_done                                                2
2795
  metaslab_class_get_dspace                                         2
2796
  metaslab_prefetch                                                 2
2797
  metaslab_sync_reassess                                            2
2798
  spa_async_dispatch                                                2
2799
  spa_deflate                                                       2
2800
  spa_handle_ignored_writes                                         2
2801
  spa_last_synced_txg                                               2
2802
  spa_sync                                                          2
2803
  spa_sync_upgrades                                                 2
2804
  spa_update_dspace                                                 2
2805
  txg_dispatch_callbacks                                            2
2806
  txg_quiesce                                                       2
2807
  uberblock_update                                                  2
2808
  vdev_config_sync                                                  2
2809
  vdev_label_sync_list                                              2
2810
  vdev_sync_done                                                    2
2811
  vdev_writeable                                                    2
2812
  zio_flush                                                         2
2813
  dmu_objset_do_userquota_updates                                   3
2814
  spa_set_rootblkptr                                                3
2815
  vdev_sync                                                         3
2816
  vdev_uberblock_sync                                               3
2817
  ddt_sync                                                          4
2818
  dsl_dir_willuse_space                                             4
2819
  dsl_pool_sync                                                     4
2820
  dsl_pool_willuse_space                                            4
2821
  dsl_scan_sync                                                     4
2822
  spa_errlog_sync                                                   4
2823
  spa_sync_config_object                                            4
2824
  vdev_disk_ioctl_done                                              4
2825
  vdev_disk_ioctl_free                                              4
2826
  vdev_readable                                                     4
2827
  zap_increment                                                     4
2828
  zap_increment_int                                                 4
2829
  zap_update                                                        4
2830
  dmu_objset_sync                                                   5
2831
  dmu_objset_write_done                                             5
2832
  dmu_objset_write_ready                                            5
2833
  zil_header_in_syncing_context                                     5
2834
  zil_sync                                                          5
2835
  arc_adjust                                                        6
2836
  arc_reclaim_needed                                                6
2837
  dmu_bonus_hold                                                    6
2838
  metaslab_class_space_update                                       6
2839
  metaslab_sync_done                                                6
2840
  space_map_load_wait                                               6
2841
  space_map_walk                                                    6
2842
  vdev_space_update                                                 6
2843
  zio_ioctl                                                         6
2844
  bplist_iterate                                                    7
2845
  dmu_objset_is_dirty                                               7
2846
  dsl_dir_sync                                                      7
2847
  spa_sync_aux_dev                                                  8
2848
  txg_thread_wait                                                   8
2849
  vdev_label_offset                                                 8
2850
  vdev_label_write                                                  8
2851
  vdev_uberblock_sync_done                                          8
2852
  zap_lookup                                                        8
2853
  zap_lookup_norm                                                   8
2854
  zio_checksum_SHA256                                               8
2855
  zio_checksum_label_verifier                                       8
2856
  zio_write_phys                                                    8
2857
  bpobj_space                                                       9
2858
  dsl_scan_active                                                   9
2859
  metaslab_sync                                                     9
2860
  rrn_find_and_remove                                               9
2861
  rrw_enter                                                         9
2862
  rrw_enter_read                                                    9
2863
  rrw_exit                                                          9
2864
  spa_meta_objset                                                   9
2865
  spa_shutting_down                                                 9
2866
  zfs_getpage                                                       9
2867
  zfs_tstamp_update_setup                                           9
2868
  dmu_objset_sync_dnodes                                           10
2869
  add_reference                                                    12
2870
  arc_buf_add_ref                                                  12
2871
  arc_buf_remove_ref                                               12
2872
  dbuf_unoverride                                                  12
2873
  dmu_buf_hold                                                     12
2874
  dmu_buf_hold_array                                               12
2875
  dmu_buf_hold_array_by_dnode                                      12
2876
  dmu_buf_rele_array                                               12
2877
  dmu_object_info_from_db                                          12
2878
  dmu_object_info_from_dnode                                       12
2879
  dmu_objset_spa                                                   12
2880
  dmu_write                                                        12
2881
  dsl_dir_space_towrite                                            12
2882
  dsl_dir_willuse_space_impl                                       12
2883
  metaslab_df_fragmented                                           12
2884
  mze_find                                                         12
2885
  remove_reference                                                 12
2886
  space_map_sync                                                   12
2887
  space_map_vacate                                                 12
2888
  txg_list_empty                                                   12
2889
  zap_hash                                                         12
2890
  zap_hashbits                                                     12
2891
  zap_lockdir                                                      12
2892
  zap_match                                                        12
2893
  zap_name_alloc                                                   12
2894
  zap_name_free                                                    12
2895
  zap_unlockdir                                                    12
2896
  dmu_zfetch_colinear                                              13
2897
  dmu_zfetch_stream_reclaim                                        13
2898
  dbuf_sync_indirect                                               15
2899
  dmu_tx_commit                                                    16
2900
  dmu_tx_create_assigned                                           16
2901
  dmu_tx_create_dd                                                 16
2902
  vdev_dirty                                                       18
2903
  dmu_zfetch                                                       19
2904
  dmu_zfetch_find                                                  19
2905
  dnode_new_blkid                                                  19
2906
  ddt_enter                                                        20
2907
  ddt_exit                                                         20
2908
  dnode_clear_range                                                20
2909
  dnode_add_ref                                                    22
2910
  dbuf_fix_old_data                                                23
2911
  vdev_queue_agg_io_done                                           23
2912
  zio_vdev_delegated_io                                            23
2913
  bplist_append                                                    25
2914
  dnode_sync                                                       25
2915
  dsl_dataset_block_born                                           25
2916
  dsl_dataset_block_kill                                           25
2917
  dsl_free                                                         25
2918
  metaslab_alloc                                                   25
2919
  metaslab_free                                                    25
2920
  spa_free_sync_cb                                                 25
2921
  zio_dva_allocate                                                 25
2922
  zio_dva_free                                                     25
2923
  zio_free                                                         25
2924
  zio_free_bp_init                                                 25
2925
  zio_free_sync                                                    25
2926
  dbuf_release_bp                                                  27
2927
  dmu_objset_pool                                                  27
2928
  bp_get_dsize                                                     28
2929
  dbuf_block_freeable                                              28
2930
  spa_get_asize                                                    28
2931
  dmu_objset_userquota_get_ids                                     29
2932
  ddt_prefetch                                                     30
2933
  dmu_buf_get_user                                                 31
2934
  dnode_hold                                                       31
2935
  dnode_hold_impl                                                  31
2936
  dsl_dataset_dirty                                                31
2937
  arc_release_bp                                                   32
2938
  refcount_create                                                  32
2939
  refcount_destroy_many                                            32
2940
  arc_buf_freeze                                                   33
2941
  spa_normal_class                                                 33
2942
  spa_get_dsl                                                      34
2943
  arc_set_callback                                                 35
2944
  dbuf_check_blkptr                                                35
2945
  dbuf_write                                                       35
2946
  dbuf_write_done                                                  35
2947
  dbuf_write_ready                                                 35
2948
  dsl_pool_sync_context                                            35
2949
  dbuf_sync_leaf                                                   36
2950
  dnode_diduse_space                                               36
2951
  zio_root                                                         38
2952
  zio_wait                                                         38
2953
  arc_write                                                        40
2954
  arc_write_done                                                   40
2955
  arc_write_ready                                                  40
2956
  buf_hash_insert                                                  40
2957
  buf_hash_remove                                                  40
2958
  dbuf_sync_list                                                   40
2959
  ddt_repair_table                                                 40
2960
  ddt_sync_table                                                   40
2961
  dmu_write_policy                                                 40
2962
  fletcher_4_native                                                40
2963
  lzjb_compress                                                    40
2964
  zio_compress_data                                                40
2965
  zio_null                                                         40
2966
  zio_write                                                        40
2967
  arc_release                                                      41
2968
  buf_discard_identity                                             41
2969
  dmu_tx_get_txg                                                   42
2970
  spa_get_random                                                   42
2971
  arc_referenced                                                   43
2972
  dbuf_find                                                        43
2973
  dbuf_hash                                                        43
2974
  dbuf_hold                                                        43
2975
  dbuf_hold_impl                                                   43
2976
  dbuf_update_data                                                 43
2977
  dbuf_whichblock                                                  43
2978
  dnode_setdirty                                                   44
2979
  dbuf_add_ref                                                     48
2980
  mze_compare                                                      48
2981
  zap_getflags                                                     48
2982
  zio_checksum_compute                                             48
2983
  zio_checksum_generate                                            48
2984
  zio_issue_async                                                  48
2985
  zio_push_transform                                               48
2986
  arc_buf_thaw                                                     49
2987
  arc_access                                                       52
2988
  dbuf_will_dirty                                                  54
2989
  dnode_rele                                                       54
2990
  dmu_tx_willuse_space                                             56
2991
  dnode_willuse_space                                              56
2992
  zio_vdev_io_reissue                                              57
2993
  dbuf_rele                                                        58
2994
  txg_list_remove                                                  59
2995
  ddt_histogram_add                                                60
2996
  dmu_objset_userused_enabled                                      60
2997
  metaslab_activate                                                64
2998
  metaslab_alloc_dva                                               64
2999
  metaslab_block_picker                                            64
3000
  metaslab_df_alloc                                                64
3001
  metaslab_free_dva                                                64
3002
  metaslab_group_alloc                                             64
3003
  space_map_alloc                                                  64
3004
  space_map_remove                                                 64
3005
  vdev_allocatable                                                 64
3006
  spa_freeze_txg                                                   65
3007
  zio_taskq_member                                                 65
3008
  zio_write_bp_init                                                66
3009
  spa_syncing_txg                                                  67
3010
  zio_unique_parent                                                67
3011
  arc_cksum_compute                                                73
3012
  zio_vdev_io_bypass                                               73
3013
  metaslab_pp_maxsize                                              76
3014
  arc_change_state                                                 85
3015
  zio_buf_alloc                                                    85
3016
  zio_buf_free                                                     85
3017
  dbuf_dirty                                                       86
3018
  dmu_tx_dirty_buf                                                 86
3019
  arc_cksum_verify                                                 89
3020
  dbuf_read                                                        91
3021
  dsl_dir_transfer_space                                           94
3022
  dnode_verify                                                    103
3023
  spa_sync_pass                                                   104
3024
  dbuf_rele_and_unlock                                            109
3025
  zio_ready                                                       117
3026
  bp_get_dsize_sync                                               120
3027
  metaslab_distance                                               120
3028
  dsl_dir_diduse_space                                            122
3029
  spa_config_enter                                                124
3030
  spa_config_exit                                                 124
3031
  fletcher_2_native                                               138
3032
  vdev_mirror_io_done                                             149
3033
  vdev_mirror_io_start                                            149
3034
  vdev_mirror_map_alloc                                           149
3035
  vdev_mirror_map_free                                            149
3036
  arc_released                                                    156
3037
  parent_delta                                                    156
3038
  dsl_dir_dirty                                                   164
3039
  space_map_add                                                   174
3040
  vdev_accessible                                                 176
3041
  vdev_cache_write                                                176
3042
  vdev_disk_io_done                                               176
3043
  vdev_disk_io_intr                                               176
3044
  vdev_queue_io_done                                              176
3045
  vdev_disk_io_start                                              180
3046
  zio_interrupt                                                   180
3047
  vdev_is_dead                                                    182
3048
  vdev_default_asize                                              192
3049
  vdev_psize_to_asize                                             192
3050
  spa_max_replication                                             193
3051
  metaslab_group_sort                                             200
3052
  metaslab_weight                                                 200
3053
  txg_list_add                                                    214
3054
  vdev_queue_io_add                                               226
3055
  vdev_queue_io_remove                                            226
3056
  zio_taskq_dispatch                                              228
3057
  dbuf_verify                                                     238
3058
  buf_hash                                                        241
3059
  refcount_count                                                  269
3060
  refcount_add                                                    295
3061
  refcount_add_many                                               295
3062
  spa_version                                                     298
3063
  vdev_queue_io                                                   306
3064
  refcount_remove                                                 308
3065
  refcount_remove_many                                            308
3066
  vdev_queue_deadline_compare                                     312
3067
  vdev_mirror_child_done                                          327
3068
  zio_vdev_child_io                                               327
3069
  zio_nowait                                                      429
3070
  spa_writeable                                                   435
3071
  dva_get_dsize_sync                                              444
3072
  vdev_queue_io_to_issue                                          456
3073
  zio_vdev_io_start                                               459
3074
  vdev_stat_update                                                467
3075
  zio_create                                                      467
3076
  zio_destroy                                                     467
3077
  zio_gang_tree_free                                              467
3078
  zio_pop_transforms                                              467
3079
  zio_vdev_io_done                                                474
3080
  zio_vdev_io_assess                                              475
3081
  zio_add_child                                                   479
3082
  zio_remove_child                                                479
3083
  zio_done                                                        499
3084
  zio_notify_parent                                               558
3085
  dmu_tx_is_syncing                                               574
3086
  vdev_lookup_top                                                 614
3087
  vdev_queue_offset_compare                                       620
3088
  metaslab_segsize_compare                                        764
3089
  zrl_add_debug                                                   778
3090
  zrl_remove                                                      778
3091
  zio_execute                                                    1105
3092
  metaslab_compare                                               1313
3093
  zio_walk_parents                                               1372
3094
  spa_config_held                                                1467
3095
  zio_inherit_child_errors                                       1868
3096
  space_map_seg_compare                                          1962
3097
  zio_wait_for_children                                          3302
3098
  ddt_stat_add                                                   3968
3099
  refcount_is_zero                                               9122
3100

    
3101
# -----------------------------------------------------------------------------
3102
# script output/run a one liner ZFS entries (60 seconds)
3103
# ----------------------------
3104

    
3105
lab10# dtrace -n 'fbt:zfs::entry { @[probefunc] = count(); }  tick-60s { exit(0); }'
3106
dtrace: description 'fbt:zfs::entry ' matched 2335 probes
3107
CPU     ID                    FUNCTION:NAME
3108
  2  65562                        :tick-60s
3109

    
3110
  bpobj_iterate                                                     1
3111
  bpobj_iterate_impl                                                1
3112
  dbuf_free_range                                                   1
3113
  dbuf_undirty                                                      1
3114
  dmu_free_range                                                    1
3115
  dmu_objset_userquota_find_data                                    1
3116
  dnode_free_range                                                  1
3117
  dnode_sync_free_range                                             1
3118
  free_blocks                                                       1
3119
  sa_hdrsize                                                        1
3120
  txg_list_head                                                     1
3121
  txg_list_next                                                     1
3122
  vdev_uberblock_sync_list                                          1
3123
  zfs_space_delta_cb                                                1
3124
  zil_clean                                                         1
3125
  zil_itxg_clean                                                    1
3126
  dmu_object_size_from_db                                           2
3127
  do_userquota_update                                               2
3128
  dsl_dataset_block_freeable                                        2
3129
  dsl_dataset_get_spa                                               2
3130
  dsl_dataset_prev_snap_txg                                         2
3131
  dsl_dataset_sync                                                  2
3132
  sa_attr_op                                                        2
3133
  sa_bulk_lookup                                                    2
3134
  sa_bulk_lookup_locked                                             2
3135
  sa_lookup_impl                                                    2
3136
  sa_object_size                                                    2
3137
  spa_last_synced_txg                                               2
3138
  vdev_label_sync_list                                              2
3139
  vdev_writeable                                                    2
3140
  zfs_access                                                        2
3141
  zfs_ace_fuid_get_flags                                            2
3142
  zfs_ace_fuid_get_mask                                             2
3143
  zfs_ace_fuid_get_type                                             2
3144
  zfs_ace_fuid_get_who                                              2
3145
  zfs_ace_fuid_size                                                 2
3146
  zfs_acl_next_ace                                                  2
3147
  zfs_acl_node_read                                                 2
3148
  zfs_acl_valid_ace_type                                            2
3149
  zfs_close                                                         2
3150
  zfs_getattr                                                       2
3151
  zfs_open                                                          2
3152
  zfs_sync                                                          2
3153
  zfs_unix_to_v4                                                    2
3154
  zfs_zaccess                                                       2
3155
  zfs_zaccess_aces_check                                            2
3156
  zfs_zaccess_common                                                2
3157
  zfs_zaccess_dataset_check                                         2
3158
  zfs_zaccess_rwx                                                   2
3159
  zio_flush                                                         2
3160
  dmu_objset_do_userquota_updates                                   3
3161
  dmu_zfetch_stream_insert                                          3
3162
  dmu_zfetch_stream_remove                                          3
3163
  metaslab_prefetch                                                 3
3164
  metaslab_sync_reassess                                            3
3165
  spa_set_rootblkptr                                                3
3166
  vdev_sync_done                                                    3
3167
  vdev_uberblock_sync                                               3
3168
  dmu_read_uio                                                      4
3169
  dsl_dir_willuse_space                                             4
3170
  dsl_pool_willuse_space                                            4
3171
  vdev_disk_ioctl_done                                              4
3172
  vdev_disk_ioctl_free                                              4
3173
  vdev_readable                                                     4
3174
  vdev_sync                                                         4
3175
  zap_increment                                                     4
3176
  zap_increment_int                                                 4
3177
  zap_update                                                        4
3178
  zfs_fuid_map_ids                                                  4
3179
  zfs_range_lock                                                    4
3180
  zfs_range_unlock                                                  4
3181
  zfs_range_unlock_reader                                           4
3182
  zfs_read                                                          4
3183
  dmu_objset_sync                                                   5
3184
  dmu_objset_write_done                                             5
3185
  dmu_objset_write_ready                                            5
3186
  zil_header_in_syncing_context                                     5
3187
  zil_sync                                                          5
3188
  dmu_bonus_hold                                                    6
3189
  space_map_walk                                                    6
3190
  zio_ioctl                                                         6
3191
  dsl_dir_sync                                                      7
3192
  vdev_label_offset                                                 8
3193
  vdev_label_write                                                  8
3194
  vdev_uberblock_sync_done                                          8
3195
  zap_lookup                                                        8
3196
  zap_lookup_norm                                                   8
3197
  zio_checksum_SHA256                                               8
3198
  zio_checksum_label_verifier                                       8
3199
  zio_write_phys                                                    8
3200
  metaslab_class_space_update                                       9
3201
  metaslab_sync_done                                                9
3202
  space_map_load_wait                                               9
3203
  vdev_space_update                                                 9
3204
  dmu_objset_sync_dnodes                                           10
3205
  zfs_fuid_map_id                                                  10
3206
  zfs_getpage                                                      10
3207
  txg_sync_waiting                                                 11
3208
  dbuf_unoverride                                                  12
3209
  ddt_get_dedup_dspace                                             12
3210
  ddt_get_dedup_histogram                                          12
3211
  ddt_get_dedup_stats                                              12
3212
  ddt_histogram_stat                                               12
3213
  dmu_buf_hold                                                     12
3214
  dmu_object_info_from_db                                          12
3215
  dmu_object_info_from_dnode                                       12
3216
  dmu_objset_spa                                                   12
3217
  dmu_write                                                        12
3218
  dsl_dir_space_towrite                                            12
3219
  dsl_dir_willuse_space_impl                                       12
3220
  dsl_pool_sync_done                                               12
3221
  metaslab_class_get_dspace                                        12
3222
  metaslab_sync                                                    12
3223
  mze_find                                                         12
3224
  spa_async_dispatch                                               12
3225
  spa_deflate                                                      12
3226
  spa_handle_ignored_writes                                        12
3227
  spa_meta_objset                                                  12
3228
  spa_sync                                                         12
3229
  spa_sync_upgrades                                                12
3230
  spa_update_dspace                                                12
3231
  space_map_sync                                                   12
3232
  txg_dispatch_callbacks                                           12
3233
  txg_quiesce                                                      12
3234
  uberblock_update                                                 12
3235
  vdev_config_sync                                                 12
3236
  zap_hash                                                         12
3237
  zap_hashbits                                                     12
3238
  zap_lockdir                                                      12
3239
  zap_match                                                        12
3240
  zap_name_alloc                                                   12
3241
  zap_name_free                                                    12
3242
  zap_unlockdir                                                    12
3243
  ddt_sync                                                         14
3244
  dmu_zfetch_streams_equal                                         14
3245
  dsl_pool_sync                                                    14
3246
  dsl_scan_sync                                                    14
3247
  metaslab_pp_free                                                 14
3248
  spa_errlog_sync                                                  14
3249
  spa_sync_config_object                                           14
3250
  space_map_free                                                   14
3251
  zfs_tstamp_update_setup                                          14
3252
  dbuf_sync_indirect                                               15
3253
  add_reference                                                    16
3254
  arc_buf_add_ref                                                  16
3255
  arc_buf_remove_ref                                               16
3256
  dmu_buf_hold_array                                               16
3257
  dmu_buf_hold_array_by_dnode                                      16
3258
  dmu_buf_rele_array                                               16
3259
  remove_reference                                                 16
3260
  bplist_iterate                                                   17
3261
  metaslab_df_fragmented                                           18
3262
  space_map_vacate                                                 18
3263
  vdev_dirty                                                       18
3264
  dnode_new_blkid                                                  19
3265
  dnode_clear_range                                                20
3266
  dmu_zfetch_colinear                                              21
3267
  dmu_zfetch_stream_reclaim                                        21
3268
  dnode_add_ref                                                    22
3269
  rrn_find_and_remove                                              22
3270
  rrw_enter                                                        22
3271
  rrw_enter_read                                                   22
3272
  rrw_exit                                                         22
3273
  dbuf_fix_old_data                                                23
3274
  vdev_queue_agg_io_done                                           23
3275
  zio_vdev_delegated_io                                            23
3276
  dmu_zfetch_find                                                  24
3277
  bplist_append                                                    25
3278
  dnode_sync                                                       25
3279
  dsl_dataset_block_born                                           25
3280
  dsl_dataset_block_kill                                           25
3281
  dsl_free                                                         25
3282
  metaslab_alloc                                                   25
3283
  metaslab_free                                                    25
3284
  spa_free_sync_cb                                                 25
3285
  zio_dva_allocate                                                 25
3286
  zio_dva_free                                                     25
3287
  zio_free                                                         25
3288
  zio_free_bp_init                                                 25
3289
  zio_free_sync                                                    25
3290
  dbuf_release_bp                                                  27
3291
  dmu_objset_is_dirty                                              27
3292
  dmu_objset_pool                                                  27
3293
  dmu_zfetch                                                       27
3294
  bp_get_dsize                                                     28
3295
  dbuf_block_freeable                                              28
3296
  spa_get_asize                                                    28
3297
  spa_sync_aux_dev                                                 28
3298
  dmu_objset_userquota_get_ids                                     29
3299
  ddt_prefetch                                                     30
3300
  dsl_dataset_dirty                                                31
3301
  arc_release_bp                                                   32
3302
  specvp_check                                                     33
3303
  arc_set_callback                                                 35
3304
  dbuf_check_blkptr                                                35
3305
  dbuf_write                                                       35
3306
  dbuf_write_done                                                  35
3307
  dbuf_write_ready                                                 35
3308
  dmu_buf_get_user                                                 35
3309
  dnode_hold                                                       35
3310
  dnode_hold_impl                                                  35
3311
  zfs_fastaccesschk_execute                                        35
3312
  zfs_lookup                                                       35
3313
  dbuf_sync_leaf                                                   36
3314
  dnode_diduse_space                                               36
3315
  arc_buf_freeze                                                   37
3316
  arc_write                                                        40
3317
  arc_write_done                                                   40
3318
  arc_write_ready                                                  40
3319
  buf_hash_insert                                                  40
3320
  buf_hash_remove                                                  40
3321
  dbuf_sync_list                                                   40
3322
  dmu_write_policy                                                 40
3323
  fletcher_4_native                                                40
3324
  lzjb_compress                                                    40
3325
  zio_compress_data                                                40
3326
  zio_write                                                        40
3327
  arc_release                                                      41
3328
  buf_discard_identity                                             41
3329
  dmu_tx_get_txg                                                   42
3330
  spa_get_dsl                                                      42
3331
  dsl_pool_sync_context                                            43
3332
  dnode_setdirty                                                   44
3333
  dmu_tx_commit                                                    46
3334
  dmu_tx_create_assigned                                           46
3335
  dmu_tx_create_dd                                                 46
3336
  spa_normal_class                                                 46
3337
  dbuf_add_ref                                                     48
3338
  mze_compare                                                      48
3339
  txg_thread_wait                                                  48
3340
  zap_getflags                                                     48
3341
  zio_checksum_compute                                             48
3342
  zio_checksum_generate                                            48
3343
  zio_issue_async                                                  48
3344
  zio_push_transform                                               48
3345
  arc_buf_thaw                                                     49
3346
  bpobj_space                                                      49
3347
  dsl_scan_active                                                  49
3348
  spa_shutting_down                                                49
3349
  arc_referenced                                                   51
3350
  dbuf_find                                                        51
3351
  dbuf_hash                                                        51
3352
  dbuf_hold                                                        51
3353
  dbuf_hold_impl                                                   51
3354
  dbuf_update_data                                                 51
3355
  dbuf_whichblock                                                  51
3356
  spa_get_random                                                   52
3357
  dbuf_will_dirty                                                  54
3358
  arc_access                                                       56
3359
  dmu_tx_willuse_space                                             56
3360
  dnode_willuse_space                                              56
3361
  zio_vdev_io_bypass                                               56
3362
  dnode_rele                                                       58
3363
  arc_adjust                                                       60
3364
  arc_reclaim_needed                                               60
3365
  dmu_objset_userused_enabled                                      60
3366
  metaslab_activate                                                64
3367
  metaslab_alloc_dva                                               64
3368
  metaslab_block_picker                                            64
3369
  metaslab_df_alloc                                                64
3370
  metaslab_free_dva                                                64
3371
  metaslab_group_alloc                                             64
3372
  space_map_alloc                                                  64
3373
  space_map_remove                                                 64
3374
  vdev_allocatable                                                 64
3375
  spa_freeze_txg                                                   65
3376
  zio_taskq_member                                                 65
3377
  dbuf_rele                                                        66
3378
  zio_write_bp_init                                                66
3379
  zio_unique_parent                                                67
3380
  txg_list_empty                                                   73
3381
  arc_cksum_compute                                                77
3382
  spa_syncing_txg                                                  77
3383
  metaslab_pp_maxsize                                              82
3384
  zio_root                                                         82
3385
  zio_wait                                                         82
3386
  zio_null                                                         84
3387
  arc_change_state                                                 85
3388
  zio_buf_alloc                                                    85
3389
  zio_buf_free                                                     85
3390
  dbuf_dirty                                                       86
3391
  dmu_tx_dirty_buf                                                 86
3392
  arc_cksum_verify                                                 89
3393
  zio_vdev_io_reissue                                              90
3394
  refcount_create                                                  92
3395
  refcount_destroy_many                                            92
3396
  dsl_dir_transfer_space                                           94
3397
  dbuf_read                                                        99
3398
  dnode_verify                                                    111
3399
  dbuf_rele_and_unlock                                            117
3400
  bp_get_dsize_sync                                               120
3401
  ddt_enter                                                       120
3402
  ddt_exit                                                        120
3403
  metaslab_distance                                               120
3404
  dsl_dir_diduse_space                                            122
3405
  txg_list_remove                                                 130
3406
  fletcher_2_native                                               138
3407
  ddt_repair_table                                                140
3408
  ddt_sync_table                                                  140
3409
  vdev_mirror_io_done                                             149
3410
  vdev_mirror_io_start                                            149
3411
  vdev_mirror_map_alloc                                           149
3412
  vdev_mirror_map_free                                            149
3413
  spa_config_enter                                                154
3414
  spa_config_exit                                                 154
3415
  parent_delta                                                    156
3416
  arc_released                                                    160
3417
  zio_ready                                                       161
3418
  dsl_dir_dirty                                                   164
3419
  space_map_add                                                   176
3420
  vdev_default_asize                                              192
3421
  vdev_psize_to_asize                                             192
3422
  spa_max_replication                                             193
3423
  vdev_accessible                                                 193
3424
  vdev_cache_write                                                193
3425
  vdev_disk_io_done                                               193
3426
  vdev_disk_io_intr                                               193
3427
  vdev_queue_io_done                                              193
3428
  vdev_disk_io_start                                              197
3429
  zio_interrupt                                                   197
3430
  vdev_is_dead                                                    199
3431
  spa_sync_pass                                                   204
3432
  txg_list_add                                                    218
3433
  vdev_queue_io_add                                               226
3434
  vdev_queue_io_remove                                            226
3435
  zio_taskq_dispatch                                              245
3436
  dbuf_verify                                                     254
3437
  buf_hash                                                        261
3438
  vdev_queue_deadline_compare                                     284
3439
  metaslab_group_sort                                             300
3440
  metaslab_weight                                                 300
3441
  vdev_mirror_child_done                                          327
3442
  zio_vdev_child_io                                               327
3443
  spa_version                                                     338
3444
  vdev_queue_io                                                   339
3445
  refcount_add                                                    354
3446
  refcount_add_many                                               354
3447
  ddt_histogram_add                                               360
3448
  refcount_count                                                  363
3449
  refcount_remove                                                 367
3450
  refcount_remove_many                                            367
3451
  zio_nowait                                                      429
3452
  dva_get_dsize_sync                                              444
3453
  zio_vdev_io_assess                                              458
3454
  zio_add_child                                                   462
3455
  zio_remove_child                                                462
3456
  spa_writeable                                                   478
3457
  zio_vdev_io_done                                                491
3458
  zio_vdev_io_start                                               492
3459
  vdev_queue_io_to_issue                                          500
3460
  vdev_stat_update                                                511
3461
  zio_create                                                      511
3462
  zio_destroy                                                     511
3463
  zio_gang_tree_free                                              511
3464
  zio_pop_transforms                                              511
3465
  zio_notify_parent                                               541
3466
  zio_done                                                        544
3467
  dmu_tx_is_syncing                                               574
3468
  vdev_lookup_top                                                 614
3469
  vdev_queue_offset_compare                                       694
3470
  zrl_add_debug                                                   808
3471
  zrl_remove                                                      808
3472
  metaslab_segsize_compare                                        953
3473
  zio_execute                                                    1166
3474
  zio_walk_parents                                               1426
3475
  spa_config_held                                                1471
3476
  metaslab_compare                                               1974
3477
  zio_inherit_child_errors                                       2044
3478
  space_map_seg_compare                                          2090
3479
  zio_wait_for_children                                          3570
3480
  refcount_is_zero                                               9189
3481
  ddt_stat_add                                                  23808
(18-18/18)