Project

General

Profile

Actions

Feature #14189

closed

want support for dd status=

Added by Robert Mustacchi about 1 month ago. Updated 24 days ago.

Status:
Closed
Priority:
Normal
Category:
cmd - userland programs
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

Other implementations have support for the dd option status= with three options status=none, status=noxfer, and status=progress. We should add support for these. Every dd implementation has a slightly different output of its statistics and progress. We implement the progress option by having it go through a similar code path that we do for siginfo, allowing us to try and get the same output format for consistency.

In addition, we make one other notable change from other implementations which is that if stderr is not a tty, then we do not emit carriage returns and instead newlines. While it isn't recommended that folks parse this format (and it is ultimately not stable), control characters in a text file probably aren't useful. Otherwise, by default each line of progress output overwrites the previous one unless the user has emitted output either by typing it on the tty or by using the siginfo/sigusr1 features.

The following shows what dd output looks like if you enable progress and then wait a few seconds, inject a few signals, and then eventually terminate it:

dd bs=1M if=/dev/zero of=/dev/null status=progress 
311584358400 (290 GiB) bytes transferred in 7.000006 secs (41 GiB/sec)
313881+0 records in
313881+0 records out
329128083456 (307 GiB) bytes transferred in 7.393318 secs (41 GiB/sec)
534473867264 (498 GiB) bytes transferred in 12.000018 secs (41 GiB/sec)
515402+0 records in
515402+0 records out
540438167552 (503 GiB) bytes transferred in 12.133880 secs (41 GiB/sec)
712791556096 (664 GiB) bytes transferred in 16.000022 secs (41 GiB/sec)^C
692405+0 records in
692405+0 records out
726039265280 (676 GiB) bytes transferred in 16.306815 secs (41 GiB/sec)

Note, we always fire this when we hit the output loop again during a copy and go as close to the second as we can. This may mean that this will not be aligned; however, it will not drift.

If you specify noxfer, here's instead what you see when you terminate dd:

$ dd bs=1M count=100 if=/dev/zero of=/dev/null status=noxfer
100+0 records in
100+0 records out

Note here there is no output of the throughput statistics. And if you specify status=none, then you'll see none of this at all.

Actions #1

Updated by Electric Monk about 1 month ago

  • Gerrit CR set to 1773
Actions #2

Updated by Robert Mustacchi 25 days ago

I tested this by doing the following for status=progress:

  • Using dd to do fixed size transfers and watching status once a second with status=progress
  • Verifying that the per-second data that we were seeing was roughly aligned to the second. That is that it didn't drift or otherwise.
  • That interleaving the printing of that with data from SIGINFO worked correctly.
  • That final stats were printed correctly and with newlines regardless of if we used CTRL+C or hit normal count termination
  • That redirecting this output to a file did not result in carriage returns entering the file
  • Allowed transfers to get into the TiB range or stay in the byte/KiB range to verify that we were nicenumming in a reasonable way.

For status=none:

  • Ran similar tests and dropped siginfo and related and verified that it did not actually show up with any status information.
  • Verified that at the end no stats or transfer info were printed. Example:
$ ./dd if=/dev/zero of=/dev/null bs=1M status=none
^C$ 
$ ./dd if=/dev/zero of=/dev/null bs=1M status=none count=5
$ 

For status=noxfer:

  • Verified that siginfo and the final termination both don't contain transfer statistics. e.g.:
$ ./dd if=/dev/zero of=/dev/null bs=1M status=noxfer
49810+0 records in
49810+0 records out
104066+0 records in
104066+0 records out
^C227793+0 records in
227793+0 records out

For bad status values or missing ones:

$ ./dd if=/dev/zero of=/dev/null bs=1M status=foobar
dd: bad argument: "foobar" 
$ ./dd if=/dev/zero of=/dev/null bs=1M status=
dd: bad argument: "" 

Note, the fact that dd doesn't say what the key is here is not helpful; however, that's something I'd like to fix in a new change since it's a broader problem with dd.

Actions #3

Updated by Electric Monk 24 days ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit c53c97f77356a767b8a3cec554ede591cf4074d9

commit  c53c97f77356a767b8a3cec554ede591cf4074d9
Author: Robert Mustacchi <rm@fingolfin.org>
Date:   2021-11-06T15:14:28.000Z

    14189 want support for dd status=
    14190 dd could include a human byte size
    Reviewed by: C Fraire <cfraire@me.com>
    Reviewed by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
    Reviewed by: Andy Fiddaman <andy@omnios.org>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Gordon Ross <gordon.w.ross@gmail.com>

Actions

Also available in: Atom PDF