Supersize io operations in Linux
This time I will tell you something that most of you really don’t care about: Max io size.
More specific, how large of an io operation I can issue to a storage system in one go. This might not make much of a difference for most people, but when you are tweaking a system for databases, distributed computing etc, there is a case for being able to tune this properly.
About a year ago I was testing storage performance on different platforms, i.e Solaris and Linux. One of the tests was to measure the throughput and number of iops when ramping one to many processes over io operation sizes (i.e 8,32,64,128,256,512,1024,2048k).
To finalize this blog post, I had to go back to some emails I sent to myself (note book-keeping), so I am lacking some of the screendumps. But the basic setup is the same on both Ubuntu and SUSE using the lpfc-driver (Emulex).
On Linux, I just could not get the box to issue larger io operations than 256k, which I found to be quite disturbing as I really tried hard to bash the system with large iops. No matter how hard I tried (using different tools to generate the IO, and pressing the enter button on my keyboard really, really hard), I just could not get iostat to show IOs larger than 256k. Since I trust iostat, I deduct that my system does not produce larger IOs.
When it comes to reading the output in this post, there are a couple of things to keep in mind so that you don’t get confused.
* Even if you start iostat with -k to show kilobytes, the avgrq-sz is shown as the number of 512 byte disk blocks, hence you need to multiply the number seen in iostat with 512 to get the io size
* Some config parameters need to be multiplied with 4kb memory page size (I will get to that later)
To get to my point, I need to show you a couple of things. For example, I can see that dd is issuing 1MB iops like this:
malu@kmg-sandbox-0001:~$ sudo strace dd if=/dev/zero of=/dev/sdc bs=1024k count=10000