Bcache Testing: Throughput
Get your wetsuit on, we're going data diving. Throughput benchmarks using IOzone on a common SATA disk, an Intel X25-E SSD, and Bcache, using the SSD to cache a single drive.
Wednesday, August 25th, 2010
Results
The results are plotted using a bar chart to make them easier to compare. However, carefully examine the y-axis since the major and minor divisions are not the same for every graph.
The plots are of the average values with error bars representing the standard deviation based on 10 runs of the same test. Each plot has four groups of four bars each with a different color. Each bar is a different block size for IOzone. The legend tells you what color corresponds to what block size. Each group of bars represents the specific journal size (16MB, 64MB, 256MB, 1GB). Finally, each chart represents one of the 13 tests. The first 6 charts are for the write tests and the last 7 charts are for read tests.
Figure 1 below is the write test for the four block sizes for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 1: Average Write Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
It is pretty obvious that the SSD is much much faster than either the plain disk or bcache. The big question is how much does bcache improve performance over the plain disk? Figure 2 below is a plot of the percent difference between the disk performance and the two bcache options, (1) bcache with CFQ IO Scheduler, and (2)bcache with the NOOP IO Scheduler. If the percent difference is positive it means bcache is faster. If it’s negative, then the plain disk is faster.
Figure 2: Percent Difference of the Averages for Write Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
The CFQ IO scheduler allows better performance for bcache over the block sizes used if we compare the averages. The NOOP IO Scheduler actually hurts performance at the 1MB size but it is slightly faster than the CFQ Scheduler at the 16MB block size. However, if you look back at Figure 1, you will see that the differences between the CFQ and NOOP IO Schedulers lie in the standard deviation range so it’s difficult to say with any certainty that one IO scheduler is better than the other.
Figure 3 below is the re-write test for the four block sizes for the disk alone, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 3: Re-Write Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
As with the write throughput case (Figure 1), the SSD is clearly faster than the disk or any variation of bcache. Figure 4 below plots the percent different of the bcache combinations relative to the disk alone to examine whether bcache gives you a performance boost.
Figure 4: Percent Difference of the Averages for ReWrite Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
The CFQ IO scheduler only provides a small percentage performance improvement for the rewrite case compared to the disk alone case (using the averages). But the NOOP IO Scheduler really helps performance for the 16MB block size case. On the other hand, it also hurts performance at the 1MB and 4MB block size cases. However, in both cases the differences lies in the standard deviation range, limiting any conclusions we might draw from the comparison.
Figure 5 below is the random write test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 5: Random Write Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD is clearly faster in this case as well but the differences are not as remarkable as the write and re-write cases (Figures 1 and 3). Figure 6 below plots the percent different of the bcache combinations relative to the disk alone to examine whether bcache gives you a performance boost.
Figure 6: Percent Difference for Random Write Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
The CFQ IO scheduler provides some increase in performance improvement relative to the plain disk for all 4 block sizes tested. But the NOOP IO Scheduler really helps performance for the 16MB block size case (about 11%). On the other hand, it also hurts performance at the 1MB and 4MB block size cases.
Figure 7 below is the record rewrite test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 7: Record Rewrite Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
This is the first case where the SSD is about the same speed as the disk or even bcache except at the large block sizes. So one should not expect the SSD to help improve performance over the plain disk. Figure 8 plots the percentage difference between the plain disk and the two bcache tests.
Figure 8: Percent Difference for Record Rewrite Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
Bcache really helps for Record Rewrite throughput. Recall that record rewrite tests the throughput when writing and re-writing to the same spot in a file. Also remember that bcache tries to use the SSD sequentially so it’s not rewriting to the same spot on the SSD every time. Consequently, for the CFQ IO Scheduler, the performance increase for the larger block sizes is remarkable. For the 16MB block case, the performance is almost 45% faster than the plain disk case. The same is basically true for the NOOP IO scheduler except that at smaller block sizes, it actually hurts performance. However, it does provide about 42% more performance at the 16MB block size.
Figure 9 below is the fwrite test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 9: Fwrite Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD is clearly faster than the disk or bcache configurations. Figure 10 below plots the percent difference between the disk alone test versus the bcache with CFQ IO Scheduler case, and the bcache with NOOP IO Scheduler case.
Figure 10: Percent Difference for fwrite Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
Interestingly, bcache has an impact on performance but there is no general trend – it all depends upon the block size tested. When using the CFQ IO scheduler, bcache gives a slight performance improvement except for the 16MB case where is causes an almost 10% drop in performance. The NOOP IO Scheduler is not as bad but only the 4MB block case gives any performance improvement.
Figure 11 below is the frewrite test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 11: Frewrite Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD is once again champion over the disk and/or bcache . Figure 12 below compares the disk alone to the bcache with CFQ IO scheduler configuration, and the bcache with NOOP IO scheduler configuration.
Figure 12: Percent Difference for frewrite Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
Bcache with the CFQ IO Scheduler consistently gives better performance than the plain disk, but the differences are small (well within the standard deviation of the tests). On the other hand, the NOOP IO Scheduler gives worse performance for the two smaller block sizes but a bit better performance for the 8MB and 16MB block size cases. However, the performance differences are not very large.
Figure 13 below is the read test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 13: Read Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD performance is much much greater than the disk and the two bcache configurations. This was not unexpected since the general read performance of SSD’s is one of real strengths of solid state devices. Figure 14 below compares the disk alone to the bcache with CFQ IO scheduler configuration, and the bcache with NOOP IO scheduler configuration.
Figure 14: Percent Difference for Read Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
Bcache with the CFQ IO scheduler doesn’t really give much performance boost over a plain disk. The performance increases is small (less than 2%) and is well within the standard deviation of the testing. The NOOP IO scheduler gives worse performance compared to the disk alone for all block sizes except 16MB. Even then the performance boost is only about 4% – well within the standard deviation of the testing.
Figure 16 below is the re-read test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 16: Re-Read Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD performance is much much greater than the disk and the two bcache configurations as was expected. Figure 17 below compares the performance of the disk alone to the bcache with CFQ IO scheduler configuration performance, and the bcache with NOOP IO scheduler configuration performance.
Figure 17: Percent Difference for Re-Read Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
Bcache with the CFQ IO scheduler doesn’t really give much performance boost over a plain disk but it is positive for all block sizes (although the differences are well within the standard deviation). The NOOP IO scheduler gives worse performance than the plain disk at the 1MB, 4MB, and 8MB block size cases (the 1MB block size case being the worst at almost -10%). But at the 16MB block size, the performance improvement is positive but again within the standard deviation of the testing.
Figure 18 below is the random read test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 18: Random Read Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD is again much faster than the disk alone or the bcache performance. Also, the performance increases as the block size increases but this is because the read behavior becomes more sequential as the block size increases. Figure 19 below compares the performance of the disk alone to the bcache with CFQ IO scheduler configuration performance, and the bcache with NOOP IO scheduler configuration performance.
Figure 19: Percent Difference for Random Read Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
Both the CFQ and the NOOP IO schedulers improve performance relative to the plain disk. But for both cases, the percentage increase is small and within the standard deviation of the testing.
Figure 20 below is the stride read test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 20: Stride Read Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD is again much faster than the disk alone or the bcache performance. Figure 21 below compares the performance of the disk alone to the bcache with CFQ IO scheduler configuration performance, and the bcache with NOOP IO scheduler configuration performance.
Figure 21: Percent Difference for Stride Read Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
Both the CFQ and the NOOP IO schedulers improve performance relative to the plain disk for the 1MB and 4MB block sizes. The CFQ IO scheduler improves performance by about 21% for the 4MB case. The NOOP IO scheduler does not increase performance nearly as much and actually hurts performance for the 8MB and 16MB cases (the CFQ IO scheduler also hurts performance for the 16MB block size case). However, in both cases the difference is inside the standard deviation of the testing.
Figure 22 below is the fread test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 22: Stride fread Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD is again much faster than the disk alone or the bcache performance. Figure 23 below compares the performance of the disk alone to the bcache with CFQ IO scheduler configuration performance, and the bcache with NOOP IO scheduler configuration performance.
Figure 23: Percent Difference for fread Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
The CFQ IO scheduler helps performance for the block sizes tested but only by a very small amount (less than 2%). The NOOP IO scheduler help performance but really only for the 8MB case. The performance for the 1MB and 4MB cases is actually less then the drive when using the NOOP IO scheduler.
Figure 24 below is the freread test for the four block sizes for the disk along, the SSD alone, bcache using the CFQ scheduler, and bcache using the NOOP IO Scheduler.
Figure 24: Stride freread Throughput (KB per second) for the Four Blocks Sizes and for the disk, SSD, bcache with CFQ, and bcache with NOOP configurations
The SSD is again much faster than the disk alone or the bcache performance. Figure 25 below compares the performance of the disk alone to the bcache with CFQ IO scheduler configuration performance, and the bcache with NOOP IO scheduler configuration performance.
Figure 25: Percent Difference for freread Throughput (%) for bcache with CFQ and bcache with NOOP vs. just the disk
The observations for the freread cases are almost identical to the fread case. The CFQ IO scheduler helps performance for the block sizes tested but only by a very small amount (less than 2%). The NOOP IO scheduler helps performance but really only for the 8MB case. The performance for the 1MB and 4MB cases is actually less then the drive when using the NOOP IO scheduler.
Summary These two new patches, bcache and flashcache use SSD’s to cache spinning disks. Both are built with SSD’s in mind and are hoping to capitalize on the performance of SSD’s relative to a single spinning disk. This article tests bcache against a plain SSD and a plain hard drive, particularly the throughput performance. It uses IOzone for testing and runs 4 different larger block sizes (1MB, 4MB, 8MB, and 16MB). The performance of bcache was tested with the default CFQ IO scheduler as well as the NOOP IO scheduler which is thought to help SSD performance since seek time is almost zero.
I compared the average performance out of 10 runs for each of the four configurations (single disk, SSD, bcache with CFQ and bcache with NOOP). Bcache didn’t appreciably change the write performance for the write and rewrite tests. The difference in averages was well within the standard deviation of the tests. For random writes, bcache was about 7-10% faster than a plain disk but again the difference is within the standard deviation.
However, for the record rewrite test, bcache gives almost a 50% boost in performance compared to the single disk for larger block sizes (particularly at 16MB).
One would have expected the read performance of bcache to be considerably higher than a disk since SSD’s are known for the outstanding read performance. However, for the read and re-read test, the performance difference between both bcache configurations and the single disk is very small – well within the standard deviation of the tests. For the random read test, bcache gave about a 7.5% increase in performance with decreased as the block size increased. But the most spectacular performance difference was for the stride read case where bcache gave about a 21% improvement in performance over a single disk (for the 4MB block size case).
For all tests, using the SSD alone produced much better performance than either the disk or bcache.
While these results are with an early version of the patch, one was hoping for a larger performance difference. Except for the record rewrite test, bcache did not really add that much performance to a single disk on these throughput tests. Let’s hope the performance is better on the IOPS and metadata testing.
Test Results As mentioned previously, there are 13 tests that are each run 10 times, there are 4 record sizes, and it was done for the disk alone, SSD alone, bcache with CFQ IO Scheduler, and bcache with NOOP IO scheduler. This makes a total of 2,080 tests that were run.
Because of the large number of tests that are run, the results are split into two groups. The first group is for the write tests: write, re-write, random write, record re-write, fwrite, refwrite. The second group is for the read tests: read, re-read, random reads, backwards read, strided read, fread, and refread. Each table below is for one of the two groups for a specific record size (1MB, 4MB, 8MB, 16MB). So that means there are 8 tables of results.
The first two tables of results are for the 1MB record size. Table 1 below presents the throughput in KB/s for the file systems for the 6 write tests for all four configurations.
Table 1 – IOzone Write Performance Results with a Record Length of 1MB, a File Size of 16GB, for all four configurations
| Configuration |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk Alone |
95,655.50
18,001.61 |
96,224.20
18,488.00 |
69,163.90
9,904.91 |
2,941,563.70
50,533.90 |
92,604.30
18,908.18 |
94,288.10
18,879.56 |
| SSD Alone |
213,636.80
5,846.60 |
215,787.8
3,372.93 |
169,863.90
35,721.92 |
2,884,615.20
85,419.32 |
215,774.10
4,022.32 |
219,770.40
741.39 |
| Bcache – CFQ IO Scheduler |
96,243.90
15,800.91 |
96,812.50
16,374.54 |
70,464.90
8,596.79 |
2,924,564.40
36,183.67 |
93,377.50
16,814.02 |
94,972.10
16,953.60 |
| Bcache-NOOP IO Scheduler |
88,584.80
12,942.89 |
91,269.90
15,385.27 |
66,651.40
9,563.05 |
2,591,731.90
54,192.32 |
90,514.60
17,085.82 |
92,864.50
18,625.65 |
Table 2 below presents the read throughput in KB/s for the configurations for the 7 read tests for a record length of 1MB.
Table 2 – IOzone read Performance Results with a Record Length of 1MB, a File Size of 16GB, for all four configurations
| Configuration |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk Alone |
90,158.90
17,451.55 |
90,234.80
17,487.98 |
51,567.00
6,334.83 |
65,517.00
10,385.68 |
51,252.90
4,691.25 |
88,365.20
17,770.95 |
88,389.40
17,794.82 |
| SSD Alone |
249,619.80
3,756,34 |
252,989.80
205.37 |
199,183.00
293.07 |
230,431.20
18,789.28 |
243,256.50
246.96 |
246,667.40
959.67 |
253,034.10
187.35 |
| Bcache – CFQ IO Scheduler |
90,802.60
15,141.65 |
90,870.60
15,180.60 |
55,527.00
5,766.56 |
71,516.20
11,746.15 |
56,391.30
5,221.31 |
89,148.60
15,765.24 |
89,158.60
15,767.96 |
| Bcache – NOOP IO Scheduler |
86,564.00
14,209.05 |
81,122.10
13,958.58 |
52,538.50
6,695.86 |
64,104.00
10,669.66 |
55,435.00
5,191.31 |
87,527.50
16,576.82 |
87,675.00
16,405.17 |
Table 3 below presents the throughput in KB/s for the file systems for the 6 write tests for all four configurations.
Table 3 – IOzone Write Performance Results with a Record Length of 4MB, a File Size of 16GB, for all four configurations
| Configuration |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk Alone |
90,460.70
19,664.84 |
91,878.80
19,924.09 |
80,903.30
15,816.66 |
2,037,962.20
141,390.75 |
92,000.50
16,444.45 |
92,565.40
16,456.99 |
| SSD Alone |
214,135.90
5,674.87 |
219,883.30
740.88 |
216,639.70
1,613.31 |
2,274,999.30
301,805.83 |
217,245.00
2,419.56 |
216,538.40
2,004.80 |
| Bcache – CFQ IO Scheduler |
92,422.50
17,587.50 |
93,354.70
18,307.61 |
84,848.00
14,623.67 |
2,499,317.30
107,254.45 |
92,459.30
15,533.18 |
93,531.60
15,192.24 |
| Bcache-NOOP IO Scheduler |
88,915.40
18,314.24 |
90,379.00
18,221.77 |
80,791.00
14,720.72 |
1,937,345.80
46,407.91 |
87,659.20
16,583.63 |
89,066.20
17,328.50 |
Table 4 below presents the read throughput in KB/s for the configurations for the 7 read tests for a record length of 4MB for all four configurations.
Table 4 – IOzone read Performance Results with a Record Length of 4MB, a File Size of 16GB, for all four configurations
| Configuration |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk Alone |
86,170.00
18,730.50 |
86,251.40
18,773.64 |
75,701.60
14,498.21 |
85,456.50
16,549.20 |
73,571.40
26,966.68 |
87,303.90
15,535.11 |
87,393.10
15,590.41 |
| SSD Alone |
246,261.00
1,146.53 |
252,999.50
166.72 |
253,023.40
1,613.31 |
283,752.50
1,221.35 |
295,494.80
972.83 |
249,512.80
3,094.76 |
253,008.20
115.19 |
| Bcache – CFQ IO Scheduler |
87,486.60
16,964.80 |
87,602.40
17,020.38 |
80,982.90
13,628.49 |
94,210.30
16,338.16 |
88,919.30
13,709.19 |
87,978.90
14,060.30 |
88,050.10
14,091.37 |
| Bcache – NOOP IO Scheduler |
85,461.40
17,330.51 |
84,091.60
16,781.34 |
76,144.30
14,057.95 |
84,096.20
15,410.20 |
81,203.70
14,580.43 |
84,462.90
16,230.33 |
83,307.33
16,756.46 |
Table 5 below presents the throughput in KB/s for the file systems for the 6 write tests for all four configurations.
Table 5 – IOzone Write Performance Results with a Record Length of 8MB, a File Size of 16GB, for all four configurations
| Configuration |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk Alone |
90,549.80
17,910.46 |
92,557.90
18,322.32 |
86,452.60
16,402.70 |
1,363,724.90
61,192.34 |
89,512.30
18,340.48 |
90,854.50
19,765.84 |
| SSD Alone |
214,921.10
2,598.18 |
213,455.50
5,277.41 |
189,854.40
20,111.53 |
1,484,101.10
74,214.70 |
215,343.20
459.99 |
214,547.50
2,906.95 |
| Bcache – CFQ IO Scheduler |
92,978.00
16,550.33 |
92,479.60
16,628.57 |
93,109.30
16,516.06 |
1,541,258.30
48,314.37 |
90,780.00
18,283.09 |
92,542.50
17,922.32 |
| Bcache-NOOP IO Scheduler |
90,790.80
18,683.12 |
92,819.50
18,535.36 |
85,260.80
15,403.80 |
1,542,950.90
46,407.91 |
93,043.90
15,625.58 |
95,199.20
15,668.41 |
Table 6 below presents the read throughput in KB/s for the configurations for the 7 read tests for a record length of 8MB for all four configurations.
Table 6 – IOzone read Performance Results with a Record Length of 8MB, a File Size of 16GB, for all four configurations
| Configuration |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk Alone |
87,459.30
17,732.39 |
87,507.40
17,755.50 |
83,415.60
15,653.04 |
92,655.50
18,020.74 |
91,202.40
16,929.23 |
85,669.30
18,602.25 |
85,713.00
18,635.37 |
| SSD Alone |
250,686.70
3,318.58 |
253,061.60
99.48 |
266,472.30
241.18 |
294,139.80
565.33 |
303,860.50
217.83 |
250,565.00
2,743.46 |
252,990.80
161.47 |
| Bcache – CFQ IO Scheduler |
88,563.50
15,720.26 |
89,311.20
15,763.14 |
86,788.90
17,931.83 |
249,995.20
16,851.94 |
97,244.50
15,610.33 |
86,900.60
16,702.43 |
86,981.30
16,758.01 |
| Bcache – NOOP IO Scheduler |
87,063.20
17,066.97 |
87,120.10
17,336.11 |
84,040.40
14,914.48 |
91,556.80
15,710.79 |
90,163.30
14,660.38 |
89,367.80
14,586.99 |
89,439.10
14,602.19 |
Table 7 below presents the throughput in KB/s for the file systems for the 6 write tests for all four configurations.
Table 7 – IOzone Write Performance Results with a Record Length of 16MB, a File Size of 16GB, for all four configurations
| Configuration |
Write
KB/s |
Re-write
(KB/s) |
Random write
(KB/s) |
Record re-write
(KB/s) |
fwrite
(KB/s) |
frewrite
(KB/s) |
| Disk Alone |
90,526.80
16,112.00 |
89,982.70
16,391.11 |
86,724.20
15,652.09 |
1,045,254.40
29,692.57 |
91,057.30
18,340.48 |
91,951.60
19,130.89 |
| SSD Alone |
212,425.30
1,438.07 |
215,223.60
3,844.79 |
208,334.30
12,248.71 |
1,492,348.30
37,580.58 |
215,400.80
506.77 |
211,252.50
5,807.61 |
| Bcache – CFQ IO Scheduler |
90,892.20
15,348.90 |
90,090.10
16,547.27 |
91,877.00
15,077.38 |
1,506,302.50
26,431.83 |
82,007.30
16,091.78 |
93,874.70
17,236.77 |
| Bcache-NOOP IO Scheduler |
91,778.00
16,289.30 |
93,982.00
17,186.37 |
89,002.20
14,881.38 |
1,475,056.60
22,668.15 |
89,226.10
17,496.45 |
93,604.20
17,620.75 |
Table 8 below presents the read throughput in KB/s for the configurations for the 7 read tests for a record length of 16MB for all four configurations.
Table 8 – IOzone read Performance Results with a Record Length of 16MB, a File Size of 16GB, for all four configurations
| Configuration |
Read
(KB/s) |
Re-read
(KB/s) |
Random read
(KB/s) |
Backwards read
(KB/s) |
Strided read
(KB/s) |
fread
(KB/s) |
freread
(KB/s) |
| Disk Alone |
85,170.00
15,925.74 |
85,263.30
15,941.34 |
87,135.50
16,272.60 |
93,474.10
17,481.81 |
99,224.90
17,285.36 |
86,618.80
17,322.53 |
86,628.40
17,350.58 |
| SSD Alone |
250,779.70
2,289.41 |
253,012.60
136.59 |
272,936.30
2,245.55 |
299,340.60
1,777.18 |
307,307.20
1,200.56 |
251,225.60
2,907.58 |
253,029.50
53.83 |
| Bcache – CFQ IO Scheduler |
86,125.10
15,099.27 |
87,888.80
15,143.52 |
89,787.40
15,988.92 |
102,019.10
18,106.09 |
97,319.00
17,271.83 |
88,072.60
16,059.94 |
88,088.00
16,073.07 |
| Bcache – NOOP IO Scheduler |
88,185.00
15,936.32 |
88,265.50
15,978.49 |
89,910.50
15,550.34 |
96,201.80
16,793.45 |
96,951.40
16,451.38 |
86,839.20
17,210.00 |
86,932.30
17,246.51 |
Jeff Layton is an Enterprise Technologist for HPC at Dell. He can be found lounging around at a nearby Frys enjoying the coffee and waiting for sales (but never during working hours).
|
|
Comments on "Bcache Testing: Throughput"
Hi Jeff, thanks for the thorough tests. I’ve been seriously considering a 50-70GB SSD as a second drive in my laptop to reap the potential benefits of BCache and similar projects.
As a BTRFS user I am very interested in the recent BTRFS patches for hot data relocation (http://lwn.net/Articles/400029/). It will certainly be interesting to see how effective these are at increasing performance for a relatively low outlay.
Irrespective of which approach gives the best result, I think that tiered storage using complementary SSDs will be a big win for everyone – particularly Linux users, and especially those with high-performance laptops that suffer most significantly from the disk I/O bottleneck.
Regards,
Paul
@Paul,
I agree with you. While Bcache and flashcache are still in their infancy the concept of using SSD’s for caching is a good one. I can imagine taking several inexpensive SSD’s, creating a RAID-0 and using that to cache a RAID-6 or something similar.
I too am watching the btrfs patches. Next concept. In fact I think it’s good enough that the authors can make it generic and put it in the vfs (at least I hope they do).
Thanks for the post!
Jeff
I believe the block size is too big for such test. When the block size >1MB, the disk gets a lot of benefit from the sequential read or write. In one words, I think this test underestimate the benefits of using ssd and bcahce to improve performance.
In this blog:
http://virtualgeek.typepad.com/virtual_geek/2010/05/emc-unified-storage-next-generation-efficiency-details.html
benefits of “FAST Cache” feature of EMC disk-arrays is shown.
“FAST Cache” uses Flash as second-level cache of traditiona disks and it seems that it gives good improvements.
The results presented in the article are very poor. I thought bcache makes a lot of sense, but the results demonstrate it is far from ready for prime time.
Try this… (copied from my wobsehts wiki, i was having a similar problem, this may solve it…)Blank PagesIf you have “blank pages” in WordPress with wp-cache turned on after you upgrade to PHP 5.1.2 – there is simple fix to solve the problem:-<ol><li>Open /wp-content/plugins/wp-cache/wp-cache-phase2.php file* in your favourite text editor, where is the domain that you’re having troubles with EG: dreamhost.com/wp-content/plugins/wp-cache/wp-cache-phase2.php</li><li>Alternatively if that file is not in that location for some reason you can issue this command to find it from the directory: find . -name wp-cache-phase2.php</li><li>Find out wp_cache_ob_end function</li><li>then inside that function find the line with: ob_end_clean(); (it should be line 219 or about)</li><li>and finally replace that line with: ob_end_flush();</li></ol>
VVs6oL meksekmlqbkj
@dbbd,
You are correct – bcache isn’t ready for production. Even the author admits this. But I think this a time where we can influence bcache and ask for changes. For example, Kent, the author, knows that bcache doesn’t have a writethrough capability now and that impacts performance as I presented.
Great time to influence possible kernel patches.
Jeff
Jeff,
I’ve read through the article a couple of times and can’t figure one thing out.
If the working set of the test you ran was 16GB, and the SSD was 64GB, then the entire working set should have fit in the SSD cache, and the cache performance should have been very close to the SSD performance (unless bcache is VERY inefficient).
What the SSD cache size in fact set to a smaller size than the working set? If so I can’t figure out where you mention this in the article.
Thanks – Tom