In the last article we reviewed the most basic RAID levels. In this article, we take things to the next level and discuss what is called "Nested RAID Levels". These concepts can provide both performance and redundancy for a data-rich world. We then look at RAID-01 and RAID-10; two of the most common Nested RAID configurations.
This particular example was taken from wikipedia. There are a total of six drives that are all the same size (120GB). This RAID-01 array has two sets of three drives each. Each set of three drives is built with RAID-0 (the lowest level) and then they are combined using RAID-1 (mirroring).
The capacity of a RAID-01 array are the fairly simple to compute. The capacity is computed as,
Capacity = (n/2) * min(disk sizes)
where n is the number of disks in the array and min(disk sizes) is the minimum common capacity across the drives (this indicates that you can use drives of different sizes). This equation also means that RAID-01 is not very capacity efficient since you lose half of the drives to gain better data redundancy (mirroring).
Examining the RAID layout for data redundancy and failures is very enlightening. In particular what happens when a drive is lost? Let’s assume drive 2 is lost (second drive from the left). Immediately the RAID-0 group that it belongs to fails. However, since we have a mirror of the RAID-0 group, we don’t lose any data (always a good thing). We can even tolerate losing disks 1 and 3 while disk 2 is down without losing any data because they are part of the first RAID-0 group. However, after losing disk 2, we cannot tolerate losing any disk in the second RAID-0 group (disk 4, disk 5, disk 6). So the redundancy of RAID-01 is one disk (worst case) and n/2 in the best case (but not likely to happen).
Table 1 below is a quick summary of RAID-01 with a few highlights.
Table 1 – RAID-01 Highlights
| Raid Level |
Pros |
Cons |
Storage Efficiency |
Minimum Number of disks |
| RAID-01 |
- Very good performance (read and write) with the exact level of performance depending upon how many drives in a RAID-0 stripe. RAID-1 results in very good read performance.
- Reasonable data redundancy (can tolerate the loss of any one disk)
- Number of disks lost with no loss of data access varies from 1 to n/2. However, the n/2 is unlikely to happen.
|
- Very poor storage efficiency (50%)
- With Nested RAID levels, more data redundancy is desired beyond the loss of one disk.
- All disks have to be used in rebuild (more on that later in article).
|
50% assuming the drives are the same size |
4 (Note: you have to use an even number of disks) |
RAID 1+0 (RAID-10)
RAID-10 is one of the most popular Nested RAID configurations and builds the configuration in the opposite sequence of RAID-01. In Figure 3 below, a minimum configuration of four disks is shown.
Figure 3: RAID-10 layout with Four Drives
For RAID-10 the configuration begins with two groups of RAID-1 pairs (we’re assuming that RAID-1 is done with only two disks). Then RAID-0 is used to combine the RAID-1 groups. Compared to RAID-01 we are using a data redundant standard RAID level first (at the lowest level), and then using a performance standard RAID level at the highest level.
The data flow for RAID-10 is fairly simple to follow. The data in chunk “A” is first striped across the two RAID-1 pairs using RAID-0 (the highest RAID level). Then each RAID pair mirrors their particular block across the two disks (RAID-1). So mirrored data blocks are closer to one another in RAID-10 than in RAID-01.
To compare directly to RAID-01 let’s look at the same six-drive configuration as Figure 2. Figure 4 below shows the RAID-10 configuration using the same six drives.
Figure 4: RAID-10 layout with Six Drives
In this case there are three pairs of RAID-1 disks that are combined with RAID-0. Comparing to RAID-01, RAID-10 first stripes the data using RAID-0 and then uses RAID-1 for data redundancy. RAID-01 does the opposite – data is mirrored first and then stripped.
The capacity of a RAID-10 array is the fairly simple to compute and is exactly the same as RAID-01. The capacity is computed as,
Capacity = (n/2) * min(disk sizes)
where n is the number of disks in the array and min(disk sizes) is the minimum common capacity across the drives (this indicates that you can use drives of different sizes). This equation also means that RAID-10 is not very capacity efficient since you lose half of the drives to gain better data redundancy (mirroring).
But what happens in RAID-10 if you lose a disk? For example, let’s again assume we lose disk 2 (second disk from the left). You can now tolerate the loss of another disk as long as it is not disk 1. You can actually lose one disk in each RAID-1 pair without losing data. However, if you lose both disks in a RAID-1 pair the RAID-0 configuration is lost. So, just like RAID-01, you can lose one disk with out losing access to the data. But you can lose up to n/2 disks without losing access to the data, IF they are correct disks (very unlikely to happen). So just like RAID-01, the redundancy of RAID-10 is one disk (worst case) and n/2 in the best case (but not likely to happen).
Table 2 below is a quick summary of RAID-10 with a few highlights.
Table 2 – RAID-10 Highlights
| Raid Level |
Pros |
Cons |
Storage Efficiency |
Minimum Number of disks |
| RAID-10 |
- Very good performance (read and write) with the exact level of performance depending upon how many pairs in a RAID-0 stripe. RAID-1 results in very good read performance.
- Reasonable data redundancy (can tolerate the loss of one disk)
- Number of disks lost with no loss of data access varies from 1 to n/2. However, the n/2 is unlikely to happen.
- Only one disk is involved in rebuild (more on that later in article).
|
- Very poor storage efficiency
- With Nested RAID levels, more data redundancy is desired beyond the loss of one disk.
|
50% assuming the drives are the same size |
4 (Note: you have to use an even number of disks) |
Comments on "Intro to Nested-RAID: RAID-01 and RAID-10"
I think you’re missing “n” in some of your total capacity calculations. e.g.
Capacity = 3/5 * capacity of single disk
should read:
Capacity = 3n/5 * capacity of single disk
no?
I think there’s something wrong. In the example of a Linux raid configuration with m=2 and f=2, the diagram shows 4 copies of each piece of data. If this is true, then the capacity would be “(n/4) * capacity of a single disk.”
In this map
Disk 1 Disk 2 Disk 3 Disk 4
—— —— —— ——
A1 A1 A2 A2
A3 A3 A4 A4
A5 A5 A6 A6
A7 A7 A8 A8
.. .. .. ..
A2 A2 A1 A1
A4 A4 A3 A3
A6 A6 A5 A5
A8 A8 A7 A7
.. .. .. ..
We have
Capacity = (num of disks/num of chunks) * capacity of single disk
Capacity = (4/4) * capacity of single disk = capacity of single disk and of course we can lost not only 2 but all 3 disk and not lose any info
May be far-map must be
Disk 1 Disk 2 Disk 3 Disk 4
—— —— —— ——
A1 A2 A3 A4
A5 A6 A7 A8
.. .. .. ..
.. A1 A2 A3
A4 A5 A6 A7
A8 .. .. ..
Then lost any folowing 2 drive – lost some info
Ex: Disk2+Disk3 -> lost A2 and A6
Disk1+Disk3 -> all data OK
PS Last map is 4/5 of 1 disk cap
Regards
Donserg
RAID 0+1 is a sign of incompetence. Look at the data layout for 0+1 and 1+0: it is identical. The only real difference between the two algorithms is that when a drive fails in a RAID 0+1, the RAID engine automatically shoots all the other drives on the same side of the mirror. That’s just plain stupid.
How it _should_ work is: if a user tries to create a RAID 0+1 volume, the RAID engine should make a RAID 1+0 volume instead.
Wasn’t RAID5 supposed to do what RAID10 does? That is, improve both speed and reliability. Hope you compare those in a future article.
RAID 5 will do 4 IO for a write.
RAID 10 will do 2 IO for a write.
RAID 10 cost more in disk space since, space efficiency is 1/n
RAID 5 is cheaper in disk space, space efficiency is 1-1/n
Performance wise, the RAID 10 is the better choice.
But very often money is the first criteria of the management …
Wikipedia has an excellent explanation on subtility between RAID types.
See http://en.wikipedia.org/wiki/RAID
The capacity and redundancy calculations seem to be very mixed-up here.
For simple RAID-10, the article is correct. The capacity of RAID-01 (which no one would ever use) and RAID-10 are both n/2, and can tolerate the loss of any one disk. If the array is made from more than 1 set of RAID-1 pairs, you can loose 1 disk from each pair – but not two disks from the same pair.
But for mdadm RAID10, the article is mostly wrong. It starts off by missing out the “offset” option as an alternative to “near” and “far” – though much less used than “near” or “far”, it is possibly slightly faster for some loads.
The efficiency of single-mirror RAID10 is always 50%, or n/2 – regardless of the number of disks. Think about it – each block of data is written to two disks. So if you have three disks, the capacity is 3/2 – not 2/3 (or 2n/3). Regardless of the number of disks, the RAID10 will tolerate the loss of any one disk. If you have enough disks (4 or more), it /may/ tolerate the loss of other disks, depending on the spread of the data.
With four-way mirroring (near-2 and far-2), the capacity is n/4 (not n/2), and it will tolerate the loss of any /3/ disks (assuming you have at least four disks). This is the point of having multiple-copy mirroring – for each mirror, it costs you space for a duplication of the data, but you get extra redundancy.
It is also possible to use three-way mirroring (such as far-3), for n/3 capacity and two disk redundancy.
The whole last section of the article (from “Linux is special”) needs a re-write, using correct information about sizes, redundancy, and speeds. The maths is so simple for RAID-10, regardless of the number of disks and the types of mirrors, that it is hard to understand how the author got this so muddled. It is good practice to read the relevant wikipedia article before writing your own article!
Your blog is pretty cool to me and your subject matter is very relevant. I was browsing around and came across something you might find interesting. I was guilty of 3 of them with my sites. “99% of blog owners are guilty of these five BIG errors”. http://bit.ly/uEMWS2 You will be suprised how easy they are to fix.
Ongoing RAID10 nightmare.
http://www.dslreports.com/front/shutdown.html
https://docs.google.com/document/d/1kll86bDn_MgWoo6Ja7oHo_yvI0SCqggEvNWwPWIcrHY/edit
Nested raid-0 by partitioning SSD multiple times will increase the number of i/o’s, allowing dispatch from multiple cpu’s and controllers – resulting in a higher queue depth – since benchmarks show higher QD results in best performance the idea would be like an engine to reach peak power (QD) and stay there during the entire process. One might say you don’t have any way to sustain that always? but there is the idea of read ahead which can be as simple as read next block if last 2 blocks are sequential as long as the queue depth is decreasing?
8 drives, 2 controllers, 2 sockets, 4 cores each socket – (or 8 controllers?) would allow 8 cpu execution threads on windows ? assuming you stripe the two ssd raids physical raid-0 into 4 logical drives per physical raid then strip them using windows software raid all back into one?
like a short stroke without the stroke – if an ssd can handle multiple reads at once faster than single – then creating logical volumes of smaller blocks would create artificial higher queue depth than a hard drive which mostly cannot read from two places at once (untrue there is a drive with more than one actuator). nested raid could use variable strip size and a smart i/o drive to create logical volumes of 4,16,64,128 and attempt to send “like” requests and use a simple QOS algorithm like a router.