Monitoring the progress through a pipe


Small linux command, but still cooler than sliced bread: pv

When performing some operations that produces output, plenty of it, I sometimes wonder "how fast is it, really".

Normally, I just kick in a "time" before the command, check the size of the output and divide it by the time it took to perform the operation.

The other day, I was troubleshooting nfs on my Synology DS1511+, as I suspected it was not very responsive. This is a side note, though; it was slow for a datastore on my ESXi-box, as I had forgotten to enable asynchronous in the NFS rule setting for that share, so I had only 6MB/sec write rate. So, it was only slow on my ESXi-box, useless for storing virtual machines. All other use cases were ok. Turned on asynchronous in the shared folder/NFS Privileges, which solved the problem.

Back to pv and my examples. First, this is what I used to do:

malu@kmg-sandbox-0001:/mnt/synology02/files$ ls -la testfile
-rw-r--r-- 1 malu malu 448790528 2011-09-16 02:50 testfile
malu@kmg-sandbox-0001:/mnt/synology02/files$ time cat testfile > /dev/null

real 0m3.914s
user 0m0.020s
sys 0m0.910s

So, the ca 450 MB was transferred in 4 seconds, just over 100MB/sec. Not bad over a 1Gbit ethernet fileshare. But what if I want to see this during the transfer? Enter pv:

<pre>malu@sandbox $ time cat testfile | pv -b -r  > /dev/null
 428MB [ 110MB/s]
real    0m3.902s
user    0m0.040s
sys    0m1.270s</pre>

The parameters:

pv simply eats stdin, counts the bytes and sends the stream to stdout without modifying it. It cost a couple cpu-cycles, but gives me some well needed info during the transfer. You can also tell you the size of the expected stream, and it will kick in a progress bar for you as well.