Monitoring the progress through a pipe


Small linux command, but still cooler than sliced bread: pv

When performing some operations that produces output, plenty of it, I sometimes wonder “how fast is it, really”.

Normally, I just kick in a “time” before the command, check the size of the output and divide it by the time it took to perform the operation.

The other day, I was troubleshooting nfs on my Synology DS1511+, as I suspected it was not very responsive. This is a side note, though; it _was_ slow for a datastore on my ESXi-box, as I had forgotten to enable asynchronous in the NFS rule setting for that share, so I had only 6MB/sec write rate. So, it was only slow on my ESXi-box, useless for storing virtual machines. All other use cases were ok. Turned on asynchronous in the shared folder/NFS Privileges, which solved the problem.

Back to pv and my examples. First, this is what I used to do:

[ccne]

malu@kmg-sandbox-0001:/mnt/synology02/files$ ls -la testfile
-rw-r–r– 1 malu malu 448790528 2011-09-16 02:50 testfile
malu@kmg-sandbox-0001:/mnt/synology02/files$ time cat testfile > /dev/null

real 0m3.914s
user 0m0.020s
sys 0m0.910s
[/ccne]

So, the ca 450 MB was transferred in 4 seconds, just over 100MB/sec. Not bad over a 1Gbit ethernet fileshare. But what if I want to see this _during_ the transfer? Enter pv:

[ccne]

malu@sandbox $ time cat testfile | pv -b -r  > /dev/null
 428MB [ 110MB/s]

real	0m3.902s
user	0m0.040s
sys	0m1.270s

[/ccne]

The parameters:

  • -b shows the number of bytes (MB)
  • -r shows the speed/transfer-rate through the pipe
pv simply eats stdin, counts the bytes and sends the stream to stdout without modifying it. It cost a couple cpu-cycles, but gives me some well needed info during the transfer. You can also tell you the size of the expected stream, and it will kick in a progress bar for you as well.
Sweet!
Related Posts