Bcache Testing: IOPS

Previously we looked at the throughput performance of bcache by running IOzone on a common SATA disk, an Intel X25-E SSD, and Bcache using the SSD to cache a single drive. This article explores the IOPS performance of the same configuration hoping to find areas where bcache might shine.

In a recent article I presented two new patch concepts (bcache and flashcache) for improving disk performance by using SSD’s as a caching mechanism for hard drives. In reality the caching is achieved by using a block device to cache another block device, but practically it’s using an SSD to cache hard drives. I’ve been waiting for some time for a patch that increases performance by using a cache that is larger than the disk cache and faster than the disk. SSD’s fit that bill pretty well, especially given their fantastic read performance compared to disks.

The second article in the series examined the throughput performance of the bcache patches applied to the 2.6.34 kernel. The performance was not what was expected but that was actually expected (clear as mud – right?). These are early patches and don’t have the performance at this time and have not been tuned for certain workloads. Consequently, performance was not expected to be really good but there were a few bright spots in the throughput performance exploration.

This article is the next in the series and examines the IOPS performance of the four storage configurations:

  1. Single SATA II disk (7,200 rpm 500GB with 16MB cache)
  2. Single Intel X25-E SLC disk (64GB)
  3. Bcache combination that uses the Intel X25-E as a cache for the SATA drive and uses the CFQ (Completely Fair Queuing) IO Scheduler that is the default for most distributions
  4. Bcache combination that is the same as the previous but uses the NOOP IO Scheduler for the SSD that many people think could help SSD performance.

The details of the configurations are below as are the details of the benchmarks and tests run. Once again, we will be using our good benchmarking techniques in this article.

For measuring IOPS performance I’m going to use IOzone. It has switches that turn measurements into operations per second (IOPS).

IOzone and IOPS

IOzone is one of the most popular throughput benchmarks. It’s open-source and is written in very plain ANSI C (not an insult but a compliment). It is capable of single thread, multi-threaded, and multi-client testing. The basic concept of IOzone is to break up a file of a given size into records. Records are written or read in some fashion until the file size is reached. While it is more commonly used for measuring throughput performance, it can also measure operations per second (IOPS – IO Operations Per Second). More specifically, it can be used to measure sequential read and write IOPS as well as random read and random write IOPS.

For this article, IOzone runs four specific tests and computes IOPS for each. The four tests are:

  • Write
    This is a fairly simple test that simulates writing to a new file. Because of the need to create new metadata for the file, many times the writing of a new file can be slower than rewriting to an existing file. The file is written using records of a specific length (either specified by the user or chosen automatically by IOzone) until the total file length has been reached.
  • Read
    This test reads an existing file. It reads the entire file, one record at a time.
  • Random Read
    This test reads a file with the accesses being made to random locations within the file. The reads are done in record units until the total reads are the size of the file. The performance of this test is impacted by many factors including the OS cache(s), the number of disks and their configuration, disk seek latency, and disk cache among others.
  • Random Write
    The random write test measures the performance when writing a file with the accesses being made to random locations with the file. The file is opened to the total file size and then the data is written in record sizes to random locations within the file.

For IOzone the system specifications are fairly important. In particular, the amount of system memory is important because this can have a large impact on the caching effects. If the problem sizes are small enough to fit into the system or file system cache (or at least partially), it can skew results, even for IOPS testing. Comparing the results of one system where the cache effects are fairly large to a system where cache effects are not large, is comparing the proverbial apples to oranges. For example, if you run the same problem size on a system with 1GB of memory versus a system with 8GB you will get much different results.

As with the throughput tests, the IOPS tests used a file size that is twice the size of memory. The goal is to push the file size out of what could be cached by Linux. However, the actual record size within the file is under control of the user so we can effectively prescribe the size of each IO operation when performing the IOPS testing.

For this article a total file size of 16GB was used. Within this 16GB file size, five record sizes are tested: (1) 8KB, (2) 32KB, (3) 64KB, (4) 128KB, and (5) 512KB record sizes. These sizes were chosen because the run times for smaller record sizes were much longer and using our good benchmarking skills of running each test 10 times, resulted in very long benchmark times (weeks). You might laugh at the larger record sizes, but there are likely to be applications that depend upon how quickly they can read/write 512KB records (I quit saying “never” with respect to application IO – I’ve seen some truly bizarre patterns so “never” has been removed from vocabulary.).

The command line for the first record size (8KB) is,

./iozone -Rb spreadsheet_output_8K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_8K.txt

The command line for the second record size (32KB) is,

./iozone -Rb spreadsheet_output_32K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_32K.txt

The command line for the third record size (64KB) is,

./iozone -Rb spreadsheet_output_64K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_64K.txt

The command line for the fourth record size (128KB) is,

./iozone -Rb spreadsheet_output_128K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_128K.txt

The command line for the fifth record size (512KB) is,

./iozone -Rb spreadsheet_output_512K.wks -O -i 0 -i 1 -i 2 -e -+n -r 1K -s 16G > output_512K.txt

Test System

The tests were run on the same system as previous tests. The system highlights of the system are:

  • GigaByte MAA78GM-US2H motherboard
  • An AMD Phenom II X4 920 CPU
  • 8GB of memory (DDR2-800)
  • Linux 2.6.34 kernel (with bcache patches only)
  • The OS and boot drive are on an IBM DTLA-307020 (20GB drive at Ultra ATA/100)
  • /home is on a Seagate ST1360827AS
  • There are two drives for testing. They are Seagate ST3500641AS-RK with 16 MB cache each. These are /dev/sdb and /dev/sdc.

Only the second Seagate drive was used, /dev/sdc, for the file system. Since the version of bcache I used could not yet cache a block partition, I used the whole device for the file system (/dev/sdc).

Comments on "Bcache Testing: IOPS"

I truly appreciate this article post.Really looking forward to read more. Great.

Together with every little thing which seems to be developing within this subject matter, a significant percentage of perspectives are rather exciting. Nonetheless, I am sorry, because I do not subscribe to your whole suggestion, all be it exhilarating none the less. It looks to me that your remarks are not entirely justified and in reality you are generally yourself not even thoroughly convinced of the argument. In any case I did take pleasure in looking at it.

I really like and appreciate your post.Really thank you! Great.

Almost all of whatever you articulate is astonishingly accurate and it makes me wonder why I hadn’t looked at this in this light previously. This particular piece truly did turn the light on for me as far as this particular topic goes. Nonetheless there is one particular factor I am not necessarily too cozy with so whilst I make an effort to reconcile that with the central idea of the position, let me see just what all the rest of the visitors have to point out.Very well done.

I cannot thank you enough for the blog.Really looking forward to read more.

I loved your article. Cool.

Im grateful for the blog. Keep writing.

Say, you got a nice article post.Much thanks again.

Fantastic blog post.Much thanks again. Keep writing.

A lot of the things you state is supprisingly accurate and that makes me wonder why I hadn’t looked at this in this light before. This piece really did turn the light on for me personally as far as this topic goes. Nevertheless there is actually just one issue I am not really too comfortable with so while I attempt to reconcile that with the actual main idea of your point, permit me see exactly what all the rest of your readers have to point out.Well done.

Amazing! Its actually awesome article, I have got much clear idea
concerning from this post.

Check out my homepage – moving company in san diego, Milo,

In this awesome pattern of things you’ll secure an A+ just for effort and hard work. Where you confused me personally was in all the facts. As people say, the devil is in the details… And that could not be more accurate here. Having said that, let me reveal to you exactly what did do the job. Your writing is certainly highly persuasive which is most likely why I am making an effort to comment. I do not make it a regular habit of doing that. Secondly, although I can certainly see the leaps in reason you come up with, I am not convinced of how you seem to unite the ideas which in turn produce the conclusion. For the moment I will, no doubt subscribe to your point but wish in the foreseeable future you link your dots better.

I really enjoy the blog post.Thanks Again. Keep writing.

A lot of of the things you state is supprisingly appropriate and it makes me wonder the reason why I had not looked at this with this light before. This article truly did turn the light on for me personally as far as this specific topic goes. Nonetheless at this time there is actually 1 factor I am not necessarily too comfy with so while I make an effort to reconcile that with the actual central theme of your issue, permit me see exactly what all the rest of your subscribers have to point out.Nicely done.

Muchos Gracias for your article post.Really thank you!

Really appreciate you sharing this blog.Much thanks again. Cool.

That will be the finish of this report. Here you will uncover some internet sites that we assume you?ll value, just click the links.

Good way of explaining, and good post to obtain information on the topic of my presentation subject, which i am going to deliver in institution of higher education.

[url=http://canadianpharcharmyonlineusa.com/]canadian pharcharmy online[/url]
canadian pharcharmy online
http://canadianpharcharmyonlineusa.com/ canadian pharcharmy online

?f you prefer to play games online ?nside ykur t?m?, ??u m?ght aswell gget paid fo?
hav?ng fun!

?ou’ll find th??e sites t?at may pay t??i? use?s
to play activities, ?ead ? mails, ?nd join paid survey panels.

?top byy my blog post … Panda Research

Wh?n you like supplying y?ur opinion and hav? a compute?,
you c?uld w?sh t? t?y d?ing surveys
t??t are web f?om your hom? t?o prroduce a ?mall mo?e money whi?? attempting services.

Ta?e a look at my web blog :: Panda Research

Good explaining, appreciate that.

Nev?rtheless, when dealing with this k?nd off agency,
mocest and huge business will be the concentration. Those are somke of the important char?cterietics
that you want to look foor ?s y?u’r? searc?ing
for a CPA. The skills you ?cquire in yolur training to become a CPA will give
you confidence ?nn yourself ?nd your newfound

My web ?log boca Raton Wealth management

I do love the manner in which you have presented this particular difficulty plus it really does provide us some fodder for consideration. Nevertheless, because of what precisely I have seen, I simply trust as the responses stack on that individuals keep on point and not get started upon a tirade of the news du jour. All the same, thank you for this superb point and while I can not necessarily go along with it in totality, I regard your standpoint.

I do love the manner in which you have presented this specific issue and it does present me personally a lot of fodder for consideration. Nonetheless, through what I have witnessed, I just trust as other feedback pack on that people today keep on point and not start on a soap box involving the news of the day. All the same, thank you for this outstanding point and although I can not go along with it in totality, I respect your perspective.

The very root of your writing while appearing agreeable originally, did not work well with me after some time. Somewhere throughout the paragraphs you actually managed to make me a believer but only for a short while. I still have a problem with your jumps in assumptions and you would do nicely to fill in all those gaps. In the event you actually can accomplish that, I will certainly be fascinated.

Along with every thing that seems to be developing inside this specific area, your viewpoints are rather refreshing. Even so, I beg your pardon, because I do not give credence to your entire strategy, all be it radical none the less. It seems to us that your remarks are actually not completely justified and in reality you are generally yourself not really totally convinced of the point. In any event I did take pleasure in reading through it.

Fantastic blog! Do you have any recommendations
for aspiring writers? I’m hoping to start my own site soon but I’m a little lost on everything.
Would you suggest starting with a free platform like WordPress or go
for a paid option? There are so many choices out there that I’m completely confused ..

Any tips? Appreciate it!

My web blog: toxic megacolon band (Mariel)

Hello, i think that i saw you visited my website so i came to “return the favor”.I’m attempting to find
things to improve my site!I suppose its ok to use a few of your ideas!!

Also visit my webpage – toxic megacolon decompression

I do enjoy the way you have framed this matter and it really does present me personally a lot of fodder for thought. Nevertheless, because of everything that I have observed, I simply hope when the reviews pack on that men and women keep on point and don’t get started upon a tirade associated with some other news du jour. All the same, thank you for this fantastic point and while I can not necessarily go along with the idea in totality, I regard the point of view.

The other day, while I was at work, my cousin stole my iPad and tested to see
if it can survive a forty foot drop, just so she can be a youtube sensation. My iPad is now broken and she has
83 views. I know this is totally off topic but I had to share it with someone!

Also visit my web-site – wireless ceiling speakers – Wilda -

Unquestionably believe that which you said. Your favorite
justification seemed to be on the internet the simplest thing to be aware of.
I say to you, I certainly get annoyed while people consider worries that they plainly don’t know about.

You managed to hit the nail upon the top as well as defined out the whole
thing without having side-effects , people can take a signal.
Will probably be back to get more. Thanks

Here is my web page: pure forskolin extract

You c?n real?y help most of your clients by hel?ing
th?m use f?nancial statements as a basis for making
better business decisions. M?ny people are making a lot of money promoting
?PA offers, but as usual, there i? a lot of misinformation floating around.
Fo? a common judgment creditor lien to outrank an I?S lien the
lien needs to get recorded before the IRS’s
lien is recorded, and be a “Choate” lien; which m?ans it is a final lien circumstance, and that li?n specifically lists what’? attached.

m? blog; ft l?ude?dale CPA firms, Mervin,

Als?, th?y need to fulfill 150 semester hours with
33 hours of accounting classes and 36 hour? of business c?asses.
?or most sm?ll non-farm businesses, the deductions are takrn on a ?che?ule C (Prof?t or Loss From
B?siness), ?r a ?ch?dule C-EZ (for t?ose with deducti?ns of less than $5000).
There are some things peopl? ail to tnink ove? and onne of
these is dealing wit? money.

Look into my blog ost cpa in boca raton florida

If you want to take a good deal from this post then you have to apply these techniques to your won web site.

Feel free to surf to my blog post – pure forskolin extract (http://www.blablaland.com)

The very heart of your writing whilst appearing reasonable at first, did not settle properly with me personally after some time. Somewhere within the sentences you actually were able to make me a believer but just for a short while. I still have got a problem with your leaps in logic and you would do nicely to help fill in all those breaks. In the event that you can accomplish that, I could definitely end up being amazed.

I do love the way you have presented this matter and it does supply me some fodder for consideration. Nevertheless, from what I have seen, I basically trust as other opinions stack on that individuals remain on point and in no way start upon a soap box associated with the news of the day. Yet, thank you for this fantastic point and whilst I do not necessarily go along with it in totality, I regard the perspective.

I am truly glad to glance at this website posts which contains
lots of helpful information, thanks for providing these statistics.

Here is my website: forskolin reviews

Very good site you have here but I was curious about if you
knew of any message boards that cover the same topics discussed in this
article? I’d really love to be a part of community where I can get feedback from other experienced people
that share the same interest. If you have any suggestions, please let me know.
Many thanks!

My homepage – forskolin reviews (Emile)

I am not certain the place you are getting your information, but good topic.
I must spend some time finding out more or understanding
more. Thanks for great info I used to be on the lookout for this information for my mission.

My web blog pure forskolin extract (forum.bhd-clan.de)

?aking sever?l cellular CPA c?mmpaigns als makes it possible fo? you to obtain a far more
broad salary. T?e entrepreneurial owner of, say, a $2M, $5M or $20M businees
will almost ?urely be ? ver intr?guing person. Studie? ?urrently have
shown which, about average, CPAs earn 10% more
than non-CPA Accountants.

Feel free to visit my site :: boca raton wealth management group

Today, I went to the beachfront with my children. I found a sea shell and gave it to my 4 year old daughter and said
“You can hear the ocean if you put this to your ear.” She put the shell to her
ear and screamed. There was a hermit crab inside and it pinched her ear.
She never wants to go back! LoL I know this
is entirely off topic but I had to tell someone!

Look into my web site; forskolin extract

Hi Dear, are you in fact visiting this web site regularly, if so after that you will absolutely obtain fastidious

My web-site – pure forskolin extract (http://hpcgi1.nifty.com/latok_kart/bbs/g_book.cgi/RK=0/g_book.cgi)

Hi there, this weekend is pleasant for me, as this occasion i am reading this
fantastic informative post here at my residence.

My web-site; forskolin reviews

It’s remarkable in support of me to have
a web site, which is good designed for my know-how.

thanks admin

Have a look at my web page – forskolin for weight loss (Kathy)

I got this web site from my buddy who told me regarding
this web page and now this time I am browsing this
website and reading very informative posts at this place.

Look into my page: credit identity monitor; http://www.iamsport.org/pg/pages/view/29361191/,

I couldn’t resist commenting. Very well written!

My blog best credit score monitoring (ukuma.te.ua)

I blog quite often and I truly thank you for your information. The article has truly peaked my interest.
I am going to take a note of your website and keep checking for new information about once per week.
I subscribed to your RSS feed as well.

My blog post :: credit fraud australia

“I cling on to listening to the news bulletin talk about getting free online grant applications so I have been looking around for the top site to get one. Could you tell me please, where could i acquire some?”

I know this web site gives quality depending articles or reviews and
extra information, is there any other web site which presents such stuff in quality?

Look at my web site – forskolin for weight loss

Leave a Reply