dcsimg

Tools for Storage Monitoring: iostat

The world of Linux storage tools for both monitoring and management just stinks. But that doesn't mean that there is absolutely nothing to help with monitoring. One such tool, iostat, can be used to watch what your storage devices are up to.

Introductory Rant

The complete lack of good tools for managing and monitoring storage has been one of my pet peeves for some time. A recent article by Henry Newman made my simmering pot of disdain come to a complete boil. As Henry eloquently points out the network world has standards allowing the development of tools that everyone can use for monitoring and managing their networks. However, as he also points out, nothing like this exists in the storage world. Consequently, we are left with minimal tools for storage administrators to use to understand what is really happening with their storage. Moreover, given today’s world of Storage Area Networks (SANs), NFS, CIFS, iSCSI, and other network storage devices and file systems, one would think that storage tools would be integrated with network tools to provide an integrated view of what is happening with the storage solution.

However, the exact opposite is true. We have no real storage tools, just a hodge-podge of utilities that “sort-of” help us. Perhaps even worse, there is no standardization between them so a tool developed for one platform or one piece of hardware, or one file system, is usually worthless elsewhere.

We, the unwashed masses, that must deal with storage on a daily basis, and this includes home users which encompasses even those who do nothing but read Facebook and check some email, have had to suffer because of the lack of good tools, the lack of people who understand who to configure systems even for home users, and the general lack of focus on storage. As I mentioned in previous articles, the amount of storage is growing at a rate that is difficult to comprehend. This growth varies from my friend’s daughter who now has over 20,000 pictures on her laptop and in Facebook, and continues on to the gene sequencing researchers who are cranking out TB’s every week for every sequence instrument with some institutions having a very large number of sequencing instruments. Does anyone else smell a problem?

Pointing out the problem, while extremely important, is just part of actually solving it. At some point the problem needs to be tackled, wrestled, and otherwise beaten into submission. What I want to do as part of this “solution” process is to discuss existing tools that can help with storage management/monitoring. This article is the first in what is likely to be a series of articles on current storage tools for Linux. This article focuses on iostat.

iostat

IOstat is a console (text based) application that allows you to check the CPU usage as well as the device or partition performance. (Previous IOstat coverage can be found here: Quick and Dirty MySQL Performance Troubleshooting and Making Sense of System Performance.) The iostat man page says the following about iostat:

iostat - Report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions.

IOstat is part of a bigger package of performance monitoring tools called sysstat that has tools such as,

As you can tell from the links, Sebastien Godard is the maintainer of the sysstats package. You should be able to find it for your distribution but if not, you can get it from the sysstat main page and build it yourself.

Using iostat is very simple and is much like vmstat or other “stat” tools for Linux. The basic command has a few options followed by the device you want to monitor, followed by two numbers. These two are numbers are (1) the time internal between output from iostat, and (2) the number of times iostat is to be run. If you leave the second number blank, iostat will continue indefinitely or until you hit ^c to stop it.

Here is a simple example of using iostat.

[laytonj@home8 ~]$ iostat -x -m /dev/md0 1 5
Linux 2.6.18-194.el5 (home8)    11/13/2010

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
         30.67    1.91    6.49    1.08    0.00   59.84

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md0               0.00     0.00  1.89  6.03     0.02     0.02    11.23     0.00    0.00   0.00   0.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          7.00    0.00    2.00    0.00    0.00   91.00

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md0               0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          7.00    0.00    2.00    0.00    0.00   91.00

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md0               0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          7.00    0.00    4.00    0.00    0.00   89.00

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md0               0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
         12.00    0.00    5.00    0.00    0.00   83.00

Device:         rrqm/s   wrqm/s   r/s   w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await  svctm  %util
md0               0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

In this simple example I used the following options:

  • I used the extended output option (“-x”)
  • I chose to have the output appear in megabytes (“-m”)
  • I chose to have iostat report the statistics for the device, /dev/md0
  • report the values in 1 second intervals and do that 5 times.

Let’s go over the output to understand what iostat is doing.

Comments on "Tools for Storage Monitoring: iostat"

Leave a Reply