Bcache Testing: IOPS

Previously we looked at the throughput performance of bcache by running IOzone on a common SATA disk, an Intel X25-E SSD, and Bcache using the SSD to cache a single drive. This article explores the IOPS performance of the same configuration hoping to find areas where bcache might shine.

For the most part, both IO Schedulers hurt performance (basically bcache doesn’t have good random IOPS performance at this time). The CFQ IO Scheduler helps performance a little for the 8KB and 512KB record size cases. However at the other record sizes, CFQ hurts performance particularly the 128KB record size case where bcache performance is about 18% worse than the plain disk.

On the other hand, the performance using the NOOP IO Scheduler is pretty much uniformly worse for all record sizes. The worst case is for the 128KB record size where the performance is about 15% worse than the plain disk case.

Summary

The IOPS test are interesting because there are times when the bcache configurations are better than a single disk. However, overall, the SSD has much better IOPS performance which was expected due to the low seek time of the device. For example, in the case of Random Read IOPS, the SSD performance was almost 40 times greater than the plain disk or the bcache configurations. The interesting part of this exploration is to compare the two bcache configurations to the single disk (no SSD cache).

In general, the bcache configurations had fairly good performance gains over the uncached disk for sequential IOPS testing. But for random IOPS testing, the performance was rather poor. However, this was expected since the bcache wiki mentions that it’s random performance is not likely to be very good.

For the Sequential Write IOPS testing, bcache with the CFQ IO Scheduler didn’t produce much change in performance. On the other hand, bcache with the NOOP IO Scheduler did produce changes, but sometimes not for the best. For example, at 128KB record size (which is the same as the bcache block size), bcache was about 28% faster than the uncached disk, and at 8KB record size, bcache was about 9% faster. On the other hand, at 64KB record size, the performance is about 14% worse (I guess you get the good with bad when using NOOP and the current version of bcache).

The Sequential Read IOPS performance is somewhat similar. When bcache used the CFQ IO Scheduler, there was little change in performance but all of it was positive. But when the NOOP IO Scheduler was used, at 8KB record size, the performance of bcache was about 15% than the disk alone and at a record size of 128KB (again aligned with the bcache block size) the performance is 38% better than the uncached disk. However, at 64KB, the NOOP IO Scheduler produces about 7% worse performance.

The Random IOPS performance for bcache is pretty much worse for both IO Schedulers regardless of the record size. In the case of Random Write IOPS the Intel X25-E SSD used by itself was 10 times faster than the other 3 configurations, and for Random Read IOPS, it was 40 times faster.

Moreover, the performance of bcache with either the CFQ or NOOP IO Scheduler is almost always worse then the uncached disk for any record size. For example for Random Write IOPS, bcache with the CFQ IO Scheduler is 26% worse than the uncached disk at a record size of 8KB. In the configuration of bcache with the NOOP IO Scheduler, the performance degradation reaches 45% worse at a record size of 64KB.

The situation for the Random Read IOPS test is not quite as bad but bcache with the CFQ IO Scheduler at a record size of 128KB, the performance is about 18% worse than the uncached disk. The NOOP IO Scheduler with bcache results in about 16% worse performance at a record size of 128KB.

Overall, bcache did show some performance improvement for sequential IOPS, but only for certain block sizes, particularly 128KB (the block size used in bcache configuration). On the other hand, for random IOPS testing, bcache actually hurt performance relative to the uncached disk. Moreover, the SSD had much greater random IOPS performance than the disk or bcache indicating the potential of SSD’s for IOPS performance.

Next Time

So far we’ve examined throughput and now IOPS so I think it’s safe to say that the metadata performance of the four storage configurations will be examined in the next article. It should be interesting.

Appendix

This section contains the data from the plots in tabular form in case you need or want exact values from the figures.

The first table lists the results for the Sequential Write IOPS test. Table 1 below presents the Sequential Write IOPS (in IO Operations Per Second) for the five record sizes for for all four configurations.

Table 1 – IOzone Write IOPS Performance Results with a File Size of 16GB, for all 5 record sizes, and all four configurations

Configuration 8KB
(IO Operations Per Second)
32KB
(IO Operations Per Second)
64KB
(IO Operations Per Second)
128KB
(IO Operations Per Second)
512KB
(IO Operations Per Second)
Disk Alone 10,661.30
3,018.83
2,300.60
216.44
1,470.90
71.12
578.80
171.42
202.60
136.54
SSD Alone 21,138.70
599.76
5,637.60
159.38
2,887.50
39.04
1,451.60
18.48
364.40
4.58
Bcache – CFQ IO Scheduler 10,220.60
2,730.80
2,311.80
210.24
1,431.20
210.24
587.70
146.59
201.10
123.88
Bcache-NOOP IO Scheduler 11,589.30
635.36
2,134.00
113.61
1,272.50
113.61
733.30
40.22
189.50
186.34

Table 2 below presents the Sequential Read IOPS for the file systems for the five record sizes and four configurations.

Table 2 – IOzone Sequential Read IOPS Performance Results for a File Size of 16GB, for all four configurations and all five record sizes.

Configuration 8KB
(IO Operations Per Second)
32KB
(IO Operations Per Second)
64KB
(IO Operations Per Second)
128KB
(IO Operations Per Second)
512KB
(IO Operations Per Second)
Disk Alone 10,950.90
3,132.08
2,404.00
241.97
1,551.50
87.03
604.70
185.73
206.70
139.37
SSD Alone 31,448.20
34.06
7,881.40
9.12
3,949.60
2.19
1,973.20
0.93
493.00
0.00
Bcache – CFQ IO Scheduler 10,987.90
2,908.36
2,491.40
216.29
1,550.40
81.87
612.60
166.60
206.80
126.79
Bcache-NOOP IO Scheduler 12,615.90
551.13
2,395.50
571.83
1,373.40
105.30
795.50
25.16
202.70
209.57

Table 3 below presents the Random Write IOPS for the file systems for the five record sizes and four configurations.

Table 3 – IOzone Random Write IOPS Performance Results for a File Size of 16GB, for all four configurations and all five record sizes.

Configuration 8KB
(IO Operations Per Second)
32KB
(IO Operations Per Second)
64KB
(IO Operations Per Second)
128KB
(IO Operations Per Second)
512KB
(IO Operations Per Second)
Disk Alone 365.20
22.73
308.10
8.23
287.30
2.44
206.50
24.90
111.40
33.84
SSD Alone 3,762.40
665.42
4,436.00
1,205.73
2,839.60
67.60
1,403.60
28.68
347.70
6.11
Bcache – CFQ IO Scheduler 271.60
14.61
266.80
13.97
274.50
18.85
217.20
22.29
113.70
32.14
Bcache-NOOP IO Scheduler 245.90
7.58
186.00
36.37
158.40
23.75
186.60
29.88
86.50
35.98

Table 4 below presents the Random Read IOPS for the file systems for the five record sizes and four configurations.

Table 4 – IOzone Random Read IOPS Performance Results for a File Size of 16GB, for all four configurations and all five record sizes.

Configuration 8KB
(IO Operations Per Second)
32KB
(IO Operations Per Second)
64KB
(IO Operations Per Second)
128KB
(IO Operations Per Second)
512KB
(IO Operations Per Second)
Disk Alone 141.30
16.55
131.30
1.97
121.20
1.05
107.00
16.31
66.00
3.33
SSD Alone 5,659.40
6.48
3,155.10
3.03
2,228.90
4.00
1,440.00
0.50
381.70
0.48
Bcache – CFQ IO Scheduler 146.70
12.79
129.90
1.52
110.30
1.00
88.10
10.91
71.70
4.70
Bcache-NOOP IO Scheduler 138.50
2.01
117.10
14.22
104.60
1.41
90.70
1.66
62.90
11.77
Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62