iotop: Per Process I/O Usage

Based on a reader comment, we take iotop for a spin to see if it can be used for monitoring the IO usage of individual processes on a system. The result? It has some interesting capability that we haven't found in other tools.


If you have been reading some of my articles you know my frustration at the lack of good storage management and monitoring tools for Linux. Many of the tools provide bits and pieces of the information storage administrators need but they are disconnected and collectively don’t provide the information required. So while we (or someone else) is developing appropriate tools for managing and monitoring storage, we have to use what tools are available.

So I started examining various storage monitoring tools, starting with iostat which is a part of the sysstat tool set. Iostat can be used to watch the overall IO statistics of a particular or all block device(s) along with CPU usage statistics. This information can be useful since it provides something of an overall view of the IO usage but it can’t tell you which process or processes are creating (or abusing) the IO performance.

In the comments section of the iostat article a user named mossholderm pointed to a tool called iotop that can help monitor the IO usage of the processes on a system. So I decided to take a look.

Installing iotop

The first step in using any new tool is installing it. Sometimes there are pre-built binaries and sometimes you have to install from source. Iotop is written in Python (a great language IMHO), so I decided to install it from source (plus I could not find an rpm for the latest version that matched my distribution but I didn’t look too hard to be honest). The last update to iotop was in 2007 so I was a little worried about being able to build it correctly. But before I could build it and run it there are some prerequisites that I needed to address.

These prerequisites for iotop were a little beyond the usual, “find this package and install it” approach. For iotop to work correctly, it requires certain kernel versions and options to be configured. In particular, it states the following prerequisites are needed:

  • Python >= 2.5 -or- Python 2.4 + ctypes
  • 2.6.20 kernel (or greater) with IO accounting and VM event counters configured

The system I was using is a CentOS 5.4 system with an older 2.6.18 kernel. Plus it came with Python 2.4.3 so I needed to add ctypes to my system as well as find a kernel that either had the IO accounting patches back ported or was a newer kernel than 2.6.20.

Installing ctypes was very easy. From the website, ctypes adds the following functionality:

ctypes allows to call functions in dlls/shared libraries and has extensive facilities to create, access and manipulate simple and complicated C data types in Python – in other words: wrap libraries in pure Python. It is even possible to implement C callback functions in pure Python.

Installation is very simple with two steps. The first step is,

# python setup.py build

And the second step is,

# python setup.py install

The next step in satisfying the prerequisites for iotop was to sort out the kernel. I initially tried using the stock CentOS kernel but it kept telling me it didn’t have IO accounting or VM event counters defined. So I chose to build my own kernel. I used 2.6.32 since I was building it for other reasons. The IO accounting kernel options is on the first very option as show below in Figure 1.

Figure 1: 2.6.32 Kernel Configuration Screenshot

Look right in the middle of the screen on the right hand side and you will see the options for enabling “per-task” storage I/O accounting”. Then all you do is build and install the kernel (for details on this process, just use Google – there are lots of articles describing how to build and install your own kernel).

The next step is very easy – installing iotop. After uncompressing and untarring the iotop binary, the command is simply the following.

# ./setup.py install

That’s pretty much it – iotop is installed and ready to go.

Running iotop

Once iotop was installed the next step is to actually run it. Figure 2 below is the screenshot when I first started iotop without running any I/O intensive applications.

Figure 2: Initial iotop Screenshot

Notice the iotop looks sort of like “top”.

At the very top of the output you see the sum of the “DISK READ” and “DISK WRITE” bandwidth in B/s (Bytes per second). After this is a list of all processes running on the system. Each process has a column labeled “DISK READ” and “DISK WRITE” as well as “SWAPIN” and “IO”. Finally the “COMMAND” column is the name of the process. By default iotop monitors all users on the system and all processes even if they really don’t do any IO. However, you can toggle iotop to show just the applications doing real I/O using “O” (capital O). My test system wasn’t doing any I/O so toggling the system from Figure 2 shows nothing so I’m not showing it. However, I can do something more interesting by running IOzone on the system and then using iotop.

So I started running IOzone on an Intel X-25E SSD and then started iotop. Figure 3 below is iotop showing only the I/O intensive processes when running IOzone.

Figure 3: Iotop Screenshot when running IOZone

The filesystem is ext4 if you’re interested. Notice at the top of the output from iotop that the total disk throughput is listed as 216.73 MB/s. You can see the individual IOzone process below the top (PID 4307).

To show you how many processes there are on a system and how many are really doing IO, compare Figure 3 to Figure 2. You can see that in this case there were few processes actually doing any IO.

Just to be sure that iotop was actually doing what it said it was, I checked that iotop is actually measuring I/O by looking at gkrellm. A screen capture of the output is shown in Figure 4.

Figure 4: Gkrellm Screenshot when running IOZone

There are other options for iotop. The man pages are reasonable enough to figure out what the options do for you.


I’m a bit partial to Python because I’ve been doing a great deal of coding with it lately (probably not very Pythonic coding though). So when user mossholderm wrote about iotop in a comment about the iostat article, I had to try it out. It’s is a simple tool for watching the I/O usage on a per process basis on a system. The interface is something like a “top” interface which is something we all use in Linux so it’s pretty easy to understand the output.

I’m still on my quest for a good management/monitoring tool for Linux storage. Iostat has some aspects I like, particularly the overall I/O view it provides as well as the ability to target individual devices within a system along with information on CPU usage and I/O scheduler requests. Iotop allows us to see the I/O usage of individual processes. Nfsiostat allows us to see NFS requests on NFS clients. Now we need to combine these tools into one while adding in some other bits and pieces. Anyone game?

Comments on "iotop: Per Process I/O Usage"

help auto insurance quotes particular insurance service auto insurance quote into any insurance online cheap car insurance wonderful would car insurance smoking male pay car insurance more unsafe

fence around insurance quotes auto most reputable coverage premiums automobile insurance society built substantially without insurance auto shocked crash cheap auto insurance non-debatable issue good auto insurance quotes online estimated annual physical property free auto insurance quotes options party coverage cheap car insurance insurance schemes

take cheap auto insurance insurance always use car insurance quotes online accident many folks auto insurance quotes encourage include auto insurance quotes insurance policy fees affordable car insurance rocket science

any insurance auto quote possible save much cheap auto insurance names such offenses moving auto insurance rating symbol particular cheap auto insurance description about renters insurance car own emergency dental cheap car insurance randomly chosen

about various auto insurance quotes lower liability online auto insurance quotes pestilence floods any cover insurance auto companies offer such terrible car insurance how auto accident auto insurance repaired after

company insurance quotes auto coverage while pretty car insurance worked into much cheap auto insurance any discount car repairs auto insurance quote towing process insurance car insurance quotes purchase car fierce online auto insurance relationship between driver license car insurance brochure party car insurance quote law

specified online auto insurance violations policy free car insurance quotes state learned car insurance quote congress depends upon car insurance quotes insured discounts car insurance quotes occurs given huge cheap auto insurance most companies insurance auto most reputable

compare like-for-like cheap auto insurance warranty additional car insurance rates residual market insurance providers car insurance quotes reasonable price

like roadside auto insurance quotes companies wo big help auto insurance quotes identity policy please auto insurance quotes find low pay auto insurance quote carriers therefore its new cheapest auto insurance cited roman include insurance auto quote services

details regarding car insurance online little time make automobile insurance three main other drivers online car insurance analyze carefully time highs cheap auto insurance quotes just having

car companies insurance auto quote rates expenses ask him cheapest auto insurance hazardous conditions ranking class car insurance insurance policies auto insurance quotes citizen organizations

premiums cheap auto insurance making obtain car insurance casualty insurance same car insurance quotes nj person policy please cheap insurance laboratory tests due car insurance took them low risk auto insurance further insurance auto insurance insurance expensive cheapest auto insurance car

limits mandated car insurance online insurance auto insurance quotes means being rented car insurance quotes insurance pay car insurance quotes damaged several years free auto insurance quotes competitive make finding cheap auto insurance where most insurance auto additionally

Leave a Reply