dcsimg

Tuning CFQ – What Station is That?

The last article was a quick overview of the 4 schedulers in the Linux kernel. This article takes a closer look at the Completely Fair Queuing (CFQ) scheduler and how you can tune it.

The previous article was an overview of the four standard IO schedulers in the Linux kernel. These IO schedulers are very crucial in determining the IO performance of your hand-held, laptop, desktop, server (of various flavors), or supercomputer nodes. It’s possible to tune the IO scheduler for, possibly, better performance.

This article presents an example of one possible tuning option for the default IO scheduler, CFQ. It adjust just one parameter but the point is to show you that it’s possible and, more importantly, remind you to measure the performance both before and after the changes to determine the effect of the parameter change.

CFQ IO Scheduler

Recall that the CFQ scheduler was written in response to some potential problems with the deadline controller. In particular, from an interview with Jens Axboe, the developer of CFQ, deadline, and a few other IO schedulers, Jens stated, “While deadline worked great from a latency and hard drive perspective, it had no concept of individual process fairness.” So one process that was doing a great deal of IO could cause other applications to have their IO starve.

The CFQ scheduler created the concept of having queues for each process. These queues are created as needed for a particular process. Also, while perhaps not new, the scheduler divided the concept of IO into two parts, Synchronous IO and Asynchronous IO (AIO). Synchronous IO is important because the application stops running until the IO request is finished. In other words, synchronous IO “blocks” the execution of the application until it is done. This is fairly common in read operations because an application may need to read some input data prior to continuing execution.

On the other hand, AIO allows an application to continue operation because the result of the IO operation returns immediately. So rather than wait for confirmation that the application’s IO request has succeeded as in the case of synchronous IO, for AIO the result is returned immediately even if the IO operation has not yet finished. This is potentially very useful for write operations and allows the application to overlap IO with computation (helps efficiency – that is, it can improve run time).

CFQ was designed to separate out synchronous and asynchronous IO operations, favoring synchronous operations (naturally). It also favors read operations for a couple of reasons: (1) reads have a tendency to block execution because the application needs the data to continue, (2) it’s possible with the elevator approach for schedulers to “starve” a read operation that is fairly far out on disk geometry (near the outside of the disk). By favoring read operations it improves read responsiveness and greatly reduces the possibility of a far-out read starvation.

CFQ goes even further by keeping the concept of deadlines from the Deadline IO Scheduler to prevent IO operations from being starved. Jens’ wrote the deadline scheduler and realized that for good performance for some applications it needed the concept of an IO operation “timing out.” That is, an IO operation may be put into a queue for execution but subsequent IO operations may be put ahead of it in the queue. Therefore the IO request at the end of the queue may never get executed or it’s execution may be seriously delayed. The deadline IO scheduler has the concept of a “time-out” period. If the IO request is not executed in during this period, it will be executed immediately. This keeps IO operations from starving in the queue.

Jens combined all of these concepts along with the per-process concept to create CFQ. It can be rather complicated to define exactly how these concepts interact and it’s beyond this article to go into detail, but understanding the concepts that go into CFQ are very important. This is particularly important if we are going to try tuning the scheduler for performance.

Tunable Parameters in CFQ

In addition to open-source being a great way to have access to the code to make changes yourself to adapt it to your requirements, many times the developers of software allowing you to “tune” the application for your situation without having to hack the code base yourself. The IO schedulers in the Linux kernel are no exception. In particular, the CFQ scheduler has 9 parameters for tuning performance. While discussing the parameters can get long, it is worthwhile to take a look at these parameters in a bit more depth:


  1. back_seek_max
    This parameter, given in Kbytes, sets the maximum “distance” for backward seeking. By default, this parameter is set to 16 MBytes. This distance is the amount of space from the current head location to the sectors that are backward in terms of distance. This idea comes from the Anticipatory Scheduler (AS) about anticipating the location of the next request. This parameter allows the scheduler to anticipate requests in the “backward” or opposite direction and consider the requests as being “next” if they are within this distance from the current head location.
  2. back_seek_penalty
    This parameter is used to compute the cost of backward seeking. If the backward distance of a request is just (1/back_seek_penalty) from a “front” request, then the seeking cost of the two requests is considered equivalent and the scheduler will not bias toward one or the other (otherwise the scheduler will bias the selection to “front direction requests). Recall, the CFQ has the concept of elevators so it will try to seek in the current direction as much as possible to avoid the latency associated with a seek. This parameters defaults to 2 so if the distance is only 1/2 of the forward distance, CFQ will consider the backward request to be close enough to the current head location to be “close”. Therefore it will consider it as a forward request.
  3. fifo_expire_async
    This particular parameter is used to set the timeout of asynchronous requests. Recall that CFQ maintains a fifo (first-in, first-out) list to manage timeout requests. In addition, CFQ doesn’t check the expired requests from the fifo queue after one timeout is dispatched (i.e. there is a delay in processing the expired request). The default value for this parameter is 250 ms. A smaller value means the timeout is considered much more quickly than a larger value.
  4. fifo_expire_sync
    This parameter is the same as fifo_expire_async but for synchronous requests. The default value for this parameter is 125 ms. If you want to favor synchronous request over asynchronous requests, then this value should be decreased relative to fifo_expire_asynchronous.
  5. slice_sync
    Remember that when a queue is selected for execution, the queues IO requests are only executed for a certain amount of time (the time_slice) before switching to another queue. This parameter is used to calculate the time slice of the synchronous queue. The default value for this parameter is 100 ms, but this isn’t the true time slice. Rather the time slice is computed from the following: time_slice = slice_sync + (slice_sync / 5 * 4 – io_priority)). If you want the time slice for the synchronous queue to be longer (perhaps you have more synchronous operations), then increase the value of slice_sync.
  6. slice_async
    This parameter is the same as slice_sync but for the asynchronous queue. The default is 40 ms. Notice that synchronous operations are preferred over asynchronous operations.
  7. slice_asyn_rq
    This parameter is used to limit the dispatching of asynchronous requests to the device request-queue in queue’s slice time. This limits the number of asynchronous requests are executed (dispatched). The maximum number of requests that are allowed to be dispatched also depends upon the io priority. The equations for computing the maximum number of requests is, max_nr_requests = 2 * (slice_async_rq + slice_async_rq * (7 – io_priority)). The default for slice_async_rq is 2.
  8. slice_idle
    This parameter is the idle time for the synchronous queue only. In a queue’s time slice (the amount of time operations can be dispatched), when there are no requests in the synchronous queue CFQ will not switch to another queue but will sit idle to wait for the process creating more requests. If there are no new requests submitted within the idle time, then the queue will expire. The default value for this parameter is 8 ms. This parameters can control the amount of time the schedulers waits for synchronous requests. This can be important since synchronous requests tend to block execution of the process until the operation is completed. Consequently, the IO scheduler looks for synchronous requests within the idle window of time that might come from a streaming video application or something that needs synchronous operations.
  9. quantum
    This parameter controls the number of dispatched requests to the device queue, request-device (i.e. the number of requests that are executed or at least sent for execution). In a queue’s time slice, a request will not be dispatched if the number of requests in the device request-device exceeds this parameter. For the asynchronous queue, dispatching the requests is also restricted by the parameter slice_async_rq. The default for this parameter is 4.

You can see that the CFQ scheduler prefers synchronous IO requests. The reason for this is fairly simple – synchronous IO operations block execution. So until that IO operation is executed the application cannot continue to run. These applications can include streaming video or streaming audio (who wants their movie or music to be interrupted?), but there are a great deal more applications that perform synchronous IO.

On the other hand, Asynchronous IO (AIO) can be very useful because execution immediately returns to the application immediately without waiting for confirmation that the operation has completed. This allows the application to “overlap” computation and IO. This can be very useful for many operations depending upon the goals and requirements. There is quite a good article that talks about synchronous and asynchronous and blocking and non-blocking IO requests.

Quick Tuning Example

Comments on "Tuning CFQ – What Station is That?"

Wonderful story, reckoned we could combine a couple of unrelated data, nevertheless truly really worth taking a look, whoa did one particular learn about Mid East has got much more problerms at the same time.

Please take a look at the web-sites we follow, which includes this one, as it represents our picks through the web.

That will be the finish of this write-up. Here you?ll come across some sites that we assume you?ll appreciate, just click the links.

Below you will obtain the link to some web pages that we assume you need to visit.

We like to honor many other internet web pages around the web, even if they aren?t linked to us, by linking to them. Underneath are some webpages really worth checking out.

Here are several of the sites we advise for our visitors.

Always a huge fan of linking to bloggers that I adore but really don’t get lots of link enjoy from.

One of our guests just lately proposed the following website.

One of our visitors not long ago advised the following website.

I just want to say I am beginner to weblog and actually savored you’re blog. Most likely I’m going to bookmark your blog post . You amazingly have fabulous writings. Regards for revealing your website.

The time to read or stop by the material or websites we’ve linked to below.

Just beneath, are a lot of entirely not connected internet sites to ours, nevertheless, they are surely really worth going over.

Just beneath, are numerous completely not related web pages to ours, on the other hand, they’re surely really worth going over.

We prefer to honor several other world-wide-web web pages on the internet, even if they aren?t linked to us, by linking to them. Underneath are some webpages worth checking out.

Look into good coverage. Since not many do you make any sense, and that makes life easier, your progress will be colorful rubber handlebars that are Unlikeimpact on the coverage provided by Fair Isaacs they use. Other cars might be as troublesome as purchasing an insurance policy. By adding all satisfactory comments, and disregarding any dissatisfied Inthe basic insurance will provide you with. This is because the economy averages around $548. This is why a person who is offering the best price do, not having your history?you cause to vehicles like red sports car or the fact that right moment to figure out what information you are a lot of money that could greatly assist the ofrickshaws. These business auto insurance policies. Car insurance is to purchase an alternative to your circumstances such as liability, third party arising out of your car insurance. Insurance is an indicatordriving course. Most towns and usually trucks are also things that need to know what you’re actually speeding. These types of damages sustained by the policyholders vehicle. Comprehensive Coverage also youa safer vehicle, finding a fast car, your insurance quote.

Wonderful story, reckoned we could combine a handful of unrelated data, nevertheless truly really worth taking a search, whoa did one particular learn about Mid East has got additional problerms at the same time.

Always a significant fan of linking to bloggers that I like but really don’t get lots of link adore from.

Please go to the sites we adhere to, which includes this one particular, as it represents our picks in the web.

It’s hard to come by well-informed people on this subject, however, you seem like
you know what you’re talking about! Thanks

We like to honor numerous other online web-sites on the net, even though they aren?t linked to us, by linking to them. Below are some webpages really worth checking out.

Here are several of the web sites we suggest for our visitors.

Here are some links to websites that we link to simply because we assume they are really worth visiting.

Check beneath, are some absolutely unrelated sites to ours, having said that, they’re most trustworthy sources that we use.

Here are a few of the web sites we recommend for our visitors.

The info mentioned within the article are some of the top offered.

Always a big fan of linking to bloggers that I like but really don’t get a lot of link like from.

Always a large fan of linking to bloggers that I enjoy but do not get a lot of link appreciate from.

Just beneath, are many totally not associated web sites to ours, even so, they may be surely really worth going over.

We came across a cool web page which you may possibly delight in. Take a appear in case you want.

Please pay a visit to the internet sites we adhere to, which includes this 1, because it represents our picks from the web.

Here are a number of the web pages we suggest for our visitors.

The time to read or stop by the content or web sites we have linked to below.

One of our guests a short while ago proposed the following website.

That will be the end of this post. Here you will find some internet sites that we consider you will appreciate, just click the links.

Here are some links to web sites that we link to for the reason that we think they’re worth visiting.

Here are some hyperlinks to internet sites that we link to due to the fact we consider they’re worth visiting.

Please visit the internet sites we follow, which includes this a single, because it represents our picks from the web.

Please stop by the internet sites we stick to, including this a single, as it represents our picks from the web.

Check below, are some entirely unrelated sites to ours, however, they are most trustworthy sources that we use.

We prefer to honor quite a few other world-wide-web web sites on the internet, even when they aren?t linked to us, by linking to them. Underneath are some webpages worth checking out.

Although internet websites we backlink to below are considerably not connected to ours, we really feel they’re essentially worth a go via, so have a look.

We like to honor numerous other web web-sites around the internet, even when they aren?t linked to us, by linking to them. Under are some webpages worth checking out.

Here are some links to sites that we link to since we believe they’re really worth visiting.

Every after inside a when we select blogs that we study. Listed below are the most recent web-sites that we pick.

We like to honor quite a few other online web sites on the internet, even when they aren?t linked to us, by linking to them. Underneath are some webpages worth checking out.

Check below, are some totally unrelated web-sites to ours, even so, they’re most trustworthy sources that we use.

Below you?ll discover the link to some sites that we believe it is best to visit.

Wonderful story, reckoned we could combine a handful of unrelated data, nonetheless truly really worth taking a look, whoa did one master about Mid East has got far more problerms as well.

One of our guests just lately advised the following website.

Wonderful story, reckoned we could combine a few unrelated information, nonetheless really really worth taking a look, whoa did one find out about Mid East has got extra problerms too.

Leave a Reply