SandForce 1222 SSD Testing, Part 4: Detailed IOPS Analysis

In this installment of our series examining the performance of SandForce-based consumer SSDs we dig into the IOPS performance of the drive. How does it stand up to an enterprise-class SSD? Let's find out.

Introduction and Review of Initial IOPS Performance

The first two articles presented some initial performance results of a SandForce based consumer SSD. The third article was a more comprehensive examination of the throughput performance including the testing of more data compression levels and a comparison to an enterprise level SSD represented by the Intel X-25E enterprise level SSD. The reason so much time is being devoted to SandForce based SSD’s is that they use a real-time compression algorithm to improve performance and increase longevity of the SSD. What makes it even more interesting is that performance and longevity of the SSD depend upon your data (very unique and exciting concept). If your data is very compressible then you get a performance boost and a longer lasting SSD. If your data is very incompressible, then you don’t get any performance boost and the SSD will last as long as any other with the same type of cells.

The following observations were made in the first IOPS article:

  • Both the Write IOPS and Random Write IOPS results showed a fairly reasonable decrease in performance as the data became less compressible
    • The Write IOPS performance decreased by about 63% when going from 98% compressible data to 2% compressible data
    • The Random Write IOPS performance decreased by about 46% when going from 98% compressible data to 2% compressible data
  • The Read IOPS and Random Read IOPS performance also decrease with decreasing data compressibility but the magnitude of the decrease is not nearly what is for write IOPS
    • The Read IOPS performance drops by about 19.9% when going from 98% data compressibility to 2% data compressibility
    • The Random Read IOPS performance drops by about 8.9% over the range of data compressibility
  • Comparing the Sequential Write IOPS performance to the Random Write IOPS, the following was observed:
    • At 98% data compressibility the Sequential Write IOPS is better than the Random Write IOPS
    • At 50% and 2% data compressibility the Random Write IOPS performance is actually better than the Sequential Write IOPS performance. Typically the Random Write IOPS performance is always much less than the Sequential Write IOPS performance.
  • Comparing the Sequential Read IOPS performance to the Random Read IOPS, the Sequential Write IOPS performance the following was observed:
    • The Random Read IOPS is about an order of magnitude slower than the Sequential Read IOPS

The surprise in all of the testing was the Random Write IOPS performance which was just amazing. But there are some still unanswered questions about the performance. In particular, it would be nice to have more insight into how quickly the performance changes as a function of the data compressibility. Does is decrease very quickly or is it more gradual? Also, how does the performance compare to a flag ship SSD such as the 64GB Intel X-25E SSD? Is it just as good or close? Is it better?

In this article, the IOPS performance of the drive is examined more in depth and compared to the Intel X-25E (although this article is not a true product comparison). The same set of tests are run against both SSD’s on the same system to hopefully gain some insight into the performance differences.

Additional Testing

As with the previous article the intent of the expanded examination is to get a better understanding of how the SandForce SSD performs as the data compressibility changes and how this performance compares to a very high performing SSD. This article extends the work from the first IOPS article in two ways: (1) it tests more data compressibility levels, and (2) it performs all of the same tests on an Intel X-25E SSD as a comparison.

As with previous tests, IOzone is used for testing the SSD drives. IOzone is particularly useful for testing the SandForce based SSDs because it has the ability to control the compressibility level (dedupability) of the data. In addition, it’s open-source and is written in very plain ANSI C (not an insult but a compliment). It is capable of single thread, multi-threaded, and multi-client testing. The basic concept of IOzone is to break up a file of a given size into records. Records are written or read in some fashion until the file size is reached. Using this concept, IOzone has a number of tests that can be performed. The four tests listed below, are performed to measure the IOPS capability of the drives (SandForce and Intel).

  • Write IOPS
  • Read IOPS
  • Random Read IOPS
  • Random Write IOPS

You can refer to the first IOPS article for more details on these tests.

The tests were run on the same system as previous tests. The highlights of the system are:

  • CentOS 5.4 with a 2.6.32 kernel
  • GigaByte MAA78GM-US2H motherboard
  • An AMD Phenom II X4 920 CPU
  • 8GB of memory (DDR2-800)
  • The OS and boot drive are on an IBM DTLA-307020 (20GB drive at Ultra ATA/100)
  • /home is on a Seagate ST1360827AS
  • Micro Center SandForce 1222, 64GB SSD. This is mounted as /dev/sdd
  • Additionally, a 64GB Intel X-25E SSD is used, courtesy of Intel. This is mounted as /dev/sdd.
  • ext4 is used as the file system with the default options

For this article a total file size of 16GB was used. Within this 16GB file size, only one record size was used corresponding to the most common size used for testing IOPS – 4KB. The command line for this record size is,

./iozone -Rb spreadsheet_output_4K.wks -O -i 0 -i 1 -i 2 -e -+n -+w 98 -+y 98 -+C 98 -r 4K -s 16G > output_4K.txt

The options “-+w”, “-+y” and “-+C” define the how much “dedupability” there is in the data and buffers. In the above command lines “98″ means that the data can be deduplicated (compressed) 98%. Note that these options are only available with a later version of IOzone.

For this article a total of six levels of data compression were examined: (1) 98%, (2) 75%, (3) 50%, (4) 25%, (5) 2%, and (6) random. The first level corresponds to data that is very compressible (98%) and the 2% level means that the data is very incompressible. The “random” option allows IOzone to select the level of data compressibility when running the tests. It is assumed that when using the “random” level of data compressibility there will be more variability in the test results.

Results

As with the previous study, the results are plotted using a bar chart to make them easier to compare. However, carefully examine the y-axis since the major and minor divisions are not the same for every graph. The plots are of the average values with error bars representing the standard deviation based on 10 runs of the same test for the five record sizes. Each plot has six groups of 2 bars each with a different color. Each group is a specific compressibility level (e.g. 98%, 75%, etc.). Within each group there are two bars of real data – one for the SandForce SSD and one for the Intel SSD.

Figure 1 below is the Sequential Write IOPS test for the six levels of data compression with a 4KB record size for both the SandForce and Intel SSDs.

iozone_write_iops.png
Figure 1: Average Sequential Write IOPS (IO Operations Per Second) for the Six Data Compression Levels (Dedupability) for a 4KB Record Size for both the SandForce SSD and the Intel SSD

The first obvious trend that is noticed is that the Sequential Write IOPS performance for the SandForce SSD drops off as the data compressibility decreases (the red bars). At 98% compressibility the performance is about 50K Write IOPS but at 50% compressibility it is 24K Write IOPS, and at 2% compressibility it is about 18.4K Write IOPS. But the performance at 75% data compressibility is still very good – about 40K IOPS.

It is also easy to see that the Sequential Write IOPS performance for the Intel SSD does not change with data compressibility (yellow bards). The only interesting features are at 98% compressibility and at “random” compressibility that both have variation that is much larger than the other compressibility levels.

Comparing the SandForce SSD to the Intel SSD, it can be seen that for data compression levels of 98% and 75%, the SandForce SSD is just as faster or faster than the Intel SSD. For compressibility levels 50% and lower, the Intel SSD has about twice the Sequential Write IOPS performance of the SandForce SSD or greater. But remember that the Intel SSD is about $750 while the SandForce SSD is about $99.

Figure 2 below is the Read IOPS test for the six levels of data compression with a 4KB record size for both SSDs. Please note that these are Sequential Read IOPS results.

iozone_read_iops.png>
Figure 2: Average Sequential Read IOPS (IO Operations Per Second) for the Six Data Compression Levels (Dedupability) for a 4KB Record Size for both the SandForce SSD and the Intel SSD

As with the Write IOPS performance, the Read IOPS performance for the SandForce SSD does decrease with decreasing compressibility, but the relative change in performance isn’t nearly as much as it is with Write IOPS and the change isn’t “smooth”. On the other hand, the performance of the Intel SSD doesn’t change much at all as the data compressibility changes.

The Sequential Read IOPS performance of the SandForce SSD changes as a function of the data compressibility level – at 98% data compressibility, the Read IOPS performance is about 58.8K Read IOPS, at 50% data compressibility, the Read IOPS performance is about 52.4K Read IOPS, and at 2% data compressibility, the Read IOPS performance is about 47.1K Read IOPS. There is only about a 19.9% reduction in performance going from 98% data compressibility to 2% data compressibility. What is also interesting is that at a compressibility level of 2%, the Read IOPS performance is actually greater than at 25% compressibility.

Comparing the SandForce SSD performance to the Intel SSD performance it can be seen that the Intel SSD has better performance except at 98% data compressibility where it is about the the same as the SandForce SSD. However, the difference is not large, only about 28% at 25% compressibility, whereas for the Sequential Write IOPS, Intel was about twice as fast as the SandForce SSD.

Figure 3 below is the Random Write IOPS test for the six levels of data compression with a 4KB record size for both SSDs. Please note that these are Random Write IOPS results.

iozone_random_write_iops.png
Figure 3: Average Random Write IOPS (IO Operations Per Second) for the Six Data Compression Levels (Dedupability) for a 4KB Record Size for the SandForce SSD and the Intel SSD

These results are very interesting for a number of reasons. The first obvious one is that the SandForce controller has much higher performance than the Intel SSD. It is faster than Intel for every level of data compressibility – including 2%. At 98% data compressibility, the SandForce SSD is almost 5 times faster than the Intel SSD and at 2% data compressibility it is 2.8 times faster. What is interesting is that the SandForce SSD loses 54% of its Random Write IOPS performance going from 98% compressibility to 2% compressibility and yet it is still 2.8 times faster than the Intel SSD.

Random Write IOPS performance is often ignored in favor of sequential write performance (IOPS or Throughput). While the kernel and the various components do their best to “sequentialize” data when a system has a number of applications all performing I/O to various parts of the file system, the storage devices basically see random data. So random performance, particularly IOPS, can be very important in determining the overall performance of a system.

Figure 4 below is the Random Read IOPS test for the six levels of data compression with a 4KB record size for both SSDs. Please note that these are Random Read IOPS results.

iozone_random_read_iops.png
Figure 4: Average Random Read IOPS (IO Operations Per Second) for the Six Data Compression Levels (Dedupability) for a 4KB Record Size for the SandForce SSD and the Intel SSD

In the first IOPS article only the 98%, 50%, and 2% data compressibility levels were tested with the SandForce SSD. If you only examine those three results you will see that the Random Read IOPS performance for the SandForce SSD does gradually decrease but in a smooth manner. However, Figure 4 includes the 75% and 25% data compressibility cases along with the “random” case and you can see that the performance change with data compressibility levels is not smooth.

The Random Read IOPS performance of the Intel SSD does not vary much with the data compressibility level but there is more variation in the performance at 98% compressibility and at the “random” compressibility level.

Overall, the SandForce based SSD has slightly lower Random Read IOPS performance compared to the Intel SSD.

Summary

In the first IOPS article an initial examination of the IOPS performance of the SandForce SSD was performed using IOzone that allowed different levels of data compressibility to be tested. This article built on that initial study in two ways: (1) it tested more data compressibility levels (75%, 25%, and “random”), and (2) it performed all of the same tests on an Intel X-25E SSD as a comparison to an enterprise class SSD.

The results presented here are very interesting. The overall trends observed are:

  • For Sequential Write IOPS performance:
    • The Sequential Write IOPS performance of the SandForce SSD decreases with decreasing data compressibility. It decreases by about a factor of 2 from the best performance to the lowest performance.
    • The Sequential Write IOPS performance for the Intel SSD is not really affected by the data compressibility
    • The SandForce SSD is as fast or faster than the Intel SSD at 98% and 75% data compressibility. Otherwise the Intel SSD is faster – up to twice as fast as the SandForce SSD for smaller levels of compressibility.
  • For Sequential Read IOPS performance:
    • The performance of the SandForce SSD reduces with decreasing data compressibility but the reduction does not happen smoothly.
    • The performance of the Intel SSD does not change with data compressibility
    • The performance of the SandForce and the Intel SSDs are about the same for 98% data compressibility. Otherwise, the Intel SSD is better than the SandForce SSD reaching about 28.4% better performance at 25% data compressibility.
  • For the Random Write IOPS performance:
    • The performance of the SandForce SSD decreases as the data compression level decreases while the performance of the Intel SSD does not really change with data compressibility
    • The SandForce SSD has much better performance than the Intel SSD for all levels of data compressibility.
      • At 98% data compressibility, the SandForce SSD is nearly 5 times faster than the Intel SSD
      • At 2% data compressibility, the SandForce SSD is 2.8 times faster than the Intel SSD
    • Comparing the Sequential Write IOPS to the Random Write IOPS performance for the SandForce SSD, the following was observed:
      • At 98% and 75% data compressibility, Sequential Write IOPS is better than Random Write IOPS
      • At 50%, 25%, and 2% data compressibility, the Random Write IOPS is better than Sequential Write IOPS. This is unexpected as random write performance is usually much lower than sequential performance.
  • For the Random Read IOPS performance:
    • The performance of the SandForce SSD decreases with decreasing data compressibility but the decrease is not a smooth function of the data compressibility
    • The performance of the Intel SSD does not really change with data compressibility
    • At 98% data compressibility the performance of the SandForce SSD is slightly better than the Intel SSD. But at all other levels of data compressibility, the performance of the Intel SSD is better, but not by much.

As stated in the first article, SandForce has added another “knob” to the ability of tuning storage solutions – real-time data compression in the storage device (SSD). While we can’t really control turn this knob to improve performance of change the behavior of the overall storage solution, it can be used in an overall design to improve performance and longevity of SSD storage solutions. I think of this as an opportunity. We can study our data to look for opportunities that it can be compressed efficiently or we can change our applications to take advantage of SandForce controllers.

Appendix

This section contains the data from the plots in tabular form in case you need or want exact values from the figures.

The table below lists the results for all four tests, for all six compressibility levels for both SSDs.

Table 1 – IOzone IOPS Performance Results with a File Size of 16GB, a 4KB record sizes, for data compression levels of 98%, 75%, 50%, 25%, 2%, and Random, for the SandForce SSD and Intel SSDs

SSD Type Compression Level Sequential Write IOPS Random Write IOPS Sequential Read IOPS Random Read IOPS
SandForce 98% 49,729.90
2,239.04
35,971.40
1,238.71
58,847.60
235.52
7,108.70
62.07
SandForce 75% 40,201.00
2,076.55
26,518.30
312.19
51,769.50
2,690.95
6,104.80
19.20
SandForce 50% 23,921.50
1,011.77
24,666.40
1,408.00
52,404.50
463.30
6,649.60
23.71
SandForce 25% 20,514.20
1,200.36
18,557.40
751.53
44,890.40
2,054.60
5,744.90
16.26
SandForce 2% 18,415.40
720.27
19,384.60
544.93
47,144.20
258.57
6,476.10
14.91
SandForce Random 21,744.60
477.85
20,003.30
262.06
46,853.70
2,113.09
5,787.90
14.95
Intel 98% 42,027.40
2,172.38
7,304.40
1,380.60
60,551.90
3,773.99
6,435.80
385.14
Intel 75% 42,743.20
424.07
6,934.90
39.60
62,723.80
44.94
6,690.40
4.98
Intel 50% 43,007.40
523.42
6,937.80
33.36
62,730.10
68.39
6,707.60
5.31
Intel 25% 42,716.60
452.34
6,939.40
41.88
62,756.70
42.98
6,717.10
4.55
Intel 2% 42,760.10
471.97
6,915.50
36.65
62,769.30
60.98
6,719.60
5.24
Intel Random 42,383.30
1,695.19
7,313.70
1,451.99
60,607.20
3,783.09
6,548.60
375.85
Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62