Hardware-based RAID 5 has traditionally been a privilege only afforded to servers found in data centers in large enterprises — until now. With 3ware/AMCC’s latest RAID controller, the 9500 series, you get all the benefits of expensive RAID controllers and fast redundant disk storage, but at a fraction of the price of an UltraSCSI- based setup.
Instead of pricey SCSI disks, the 9500 series uses inexpensive Serial ATA drives. (Serial ATA drives have a data transfer capability of 150 MB per second, well within the neighborhood of UltraSCSI RAID in the mid-range enterprise performance level.) The 9500 series allows you to connect up to eight hard disks to it (twelve in some models), supporting RAID in 0, 1, 10, 5, 50 and single disk (JBOD) configurations.
But how well does it work? For my test, I acquired eight 160 GB Serial ATA drives from Seagate Technologies, model ST3160023AS. Each of these drives is rated at 1.5 Gbps. With disks in hand, the challenge was mounting that many drives in a single chassis — no regular desktop server chassis and power supply would do for such an obscene amount of PC storage.
After much searching and consulting with the Celestial Lords of the Direct Access Storage Device, I discovered a PC Power and Cooling (http://www.pcpowercooling.com
) steel tower monster with twelve drive bays and a custom-ordered, 510-watt redundant power supply with 8 cables that met my requirements perfectly. The PC Power and Cooling chassis was large enough to house the massive drive array, the ASUS AMD server main board I picked, a modest video card, and a CD-ROM drive, with some wattage to spare.
Once I had the proper bill of materials to actually test the 3ware/AMCC 9500, and after cabling everything inside the case (you may want to get some Teflon ties to bind up the cable runs coming out of the controller, or you’ll have a spaghetti-like mess), the setup was fairly straightforward. Like any other RAID controller, the unit has an onboard BIOS and a menu-driven system that allows you to configure the hardware RAID independent of the operating system.
I created a RAID 1 using two of the drives for the operating system and a RAID 5 of four more drives for the /home partition, keeping two of the drives in hot spare mode. (I was actually able to allocate all eight drives as a single disk RAID 5 configuration as well, but this seemed a rather impractical configuration.)
For an operating system, I installed Red Hat Enterprise Linux and used its drivers to support the controller. If you want to use a different Linux, such as a newer version of SuSE with kernel 2.6, Debian Sarge, or one of the Opteron Red Hat or SuSE distros, compile 3ware’s open source driver against your kernel and create a new initrd with the 3w_9xxx.o module in it (instructions are supplied on the 3ware web site). Optionally, you can download the sources to kernel 2.6.8 or above, which have the 3w_9xxx driver built-in and build yourself a custom kernel. Fedora Core 3, which ships with 2.6.8, supports the controller natively.
Performance of the controller is very good, but you’ll need to do some tweaking to get the most out of the bandwidth. For starters, you may want to consider using the ReiserFS or XFS filesystem for your arrays instead of ext3, because those filesystems performed better in the tests. You’ll also want to enable asynchronous reads by adding blockdev ––setra 16384 /dev/sdx to your /etc/rc.d/rc.local file (where sdx is the enumerated /dev file alias of your controller, such as /dev/sda or /dev/sdb).
Using ReiserFS and asynchronous reads, I was able to achieve transfer rates on the array volumes of about 30-40 MB per second, using 2 GB DVD and 640 MB CD-ROM ISO image files copied between array partitions. On small and medium-sized, I got somewhat higher numbers, anywhere between 40 MB and 80 MB per second.
Bottom Line: Must Buy
The 9500 series is a great value and a must buy if you’re looking to build a server with large amounts of storage, such as a big NFS box for things like demanding media content creation applications or for servers that just can’t afford to lose any data.
I also highly recommend the PC Power and Cooling Full Tower Steel Case and Turbo Cool 510w Server Power Supply (you’ll need to ask them to custom cable you a Turbo Cool with 8 SATA connectors, a “Jason Perlow Special.”) It would have been impossible to review this card at its full potential without the case and power supply. PC Power and Cooling gets an Emperor Class Award, too.
Jason Perlow eats Opterons for breakfast. Send milk to