Bcache Testing: IOPS

Previously we looked at the throughput performance of bcache by running IOzone on a common SATA disk, an Intel X25-E SSD, and Bcache using the SSD to cache a single drive. This article explores the IOPS performance of the same configuration hoping to find areas where bcache might shine.

Building ext4

For the tests in this article I used CentOS 5.4 but I used my own kernel. For all four configurations, I used the 2.6.34 kernel but with patches for bcache. I will refer to this kernel as 2.6.34+. One good thing about this kernel is that it has support for the TRIM function that most modern SSD’s have built-in. Ext4 and btrfs already have extensions to take advantage of the command.

The details of creating the file system using the disk, the SSD, and using bcache is covered in the previous article and is omitted here. So let’s turn our attention to the IOPS results.

Results

The results are plotted using bar charts to make them easier to compare. However, carefully examine the y-axis since the major and minor divisions are not the same for every graph. The plots are of the average values with error bars representing the standard deviation based on 10 runs of the same test. Each plot has four groups of five bars each with a different color representing a different record size (8KB, 32KB, 64KB, 128KB, 512KB). The legend tells you what color corresponds to what record size. Each group of bars represents a specific test (write, read, random write, random read). Finally, each chart pair represents one of the 4 tests.

Figure 1 below is the Write IOPS test for the five record sizes for the disk alone, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler. Please note that these are Sequential Write IOPS results.

iops_write.png
Figure 1: Average Sequential Write IOPS (IO Operations Per Second) for the Five Record Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations

It is pretty obvious that the SSD is much much faster in terms of IOPS than either the plain disk or bcache for all record (block) sizes. The big question is how much does bcache improve performance over the plain disk? Figure 2 below is a plot of the percent difference between the disk performance and the two bcache options, (1) bcache with CFQ IO Scheduler, and (2)bcache with the NOOP IO Scheduler. If the percent difference is positive it means bcache is faster. If it’s negative, then the plain disk is faster.

iops_write_compare.png
Figure 2: Percent Difference of the Averages for Sequential Write IOPS (%) for bcache with CFQ and bcache with NOOP vs. just the disk

The CFQ scheduler doesn’t seem to provide much change in IOPS versus the plain disk. However the NOOP scheduler can provide quite a bit of difference. At 8KB, the NOOP IO Scheduler provides about a 10% improvement over a plain disk in terms of IOPS. But at 32KB and 64KB record sizes, NOOP hurts performance by almost 15%. But the big change is at 128KB. The NOOP IO Scheduler provides about a 28% increase in Write IOPS performance over the plain disk.

Figure 3 below is the Sequential Read IOPS test for the five block sizes for the disk alone, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.

iops_read.png
Figure 3: Average Sequential Read IOPS (IO Operations Per Second) for the Five Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations

As one would expect the Read IOPS performance of the SSD is much better than the disk alone or both bcache configurations. Figure 4 below plots the percent different of the bcache combinations relative to the disk alone to examine whether bcache gives you a performance boost.

iops_read_compare.png

Figure 4: Percent Difference of the Averages for Sequential Read IOPS (%) for bcache with CFQ and bcache with NOOP vs. just the disk

The CFQ IO scheduler provides a small performance benefit – less than 5% for all of the record sizes tested. On the other hand the NOOP IO Scheduler allows some sizable improvements. For a block size of 8KB, NOOP improves performance about 15% over the disk alone. The 32KB, 64KB, and 512KB record sizes don’t show much improvement and the 64KB record size actually has worse performance (about 7%). But for the 128KB block size, the NOOP IO Scheduler improves performance by about 39%.

Figure 5 below is the Random Write IOPS Test for the five block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.

iops_random_write.png
Figure 5: Average Random Write IOPS (IO Operations Per Second) for the Five Record Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations

Here we see the performance of the Intel X25-E SSD really shine. it’s random Write IOPS performance is much greater than the other three configurations by a long stretch. Figure 6 below plots the percent different of the bcache combinations relative to the disk alone to examine whether bcache gives you a performance boost.

iops_random_write_compare.png
Figure 6: Percent Difference for Random Write IOPS (%) for bcache with CFQ and bcache with NOOP vs. just the disk

Both bcache and IO Schedulers combinations actually hurt performance compared to the plain disk – and by quite a bit. The worst case for the CFQ IO Scheduler is for the 8KB record size where is reduces performance by about 27%. The NOOP IO Scheduler is even worse reaching about 46% worse performance than the disk alone at a record size of 64KB.

Figure 7 below is a plot of the results for the Random Read IOPS test for the five block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.

iops_random_read.png
Figure 7: Average Random Read IOPS (IO Operations Per Second) for the Five Record Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations

As with the Random Write IOPS case, this case clearly shows the superiority of the Intel X25-E SSD over the plain disk or bcache. It has about 40 times greater performance for this test compared to the other three configurations (see Appendix at the end of the article). Figure 8 plots the percentage difference between the plain disk and the two bcache tests.

iops_random_read_compare.png
Figure 8: Percent Difference for Random Read IOPS (%) for bcache with CFQ and bcache with NOOP vs. just the disk

Comments on "Bcache Testing: IOPS"

Forgive my ignorance, but aren’t the record sizes in all 5 of the command-lines above set to 1k?

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>