In real estate it's about location. In storage it's about capacity. The next crop of high density drives are available but there are some gotchas related to some 3TB drives that you need to know before making a land grab.
Other Operating Systems
As previously mentioned, Linux kernels are already able to handle LBA 64-bit addressing. Linux can also handle 4KB sector drives as well. So all of the smart people using Linux are ready to use the new drives (with some caution that I will mention later) but unfortunately, “other” operating systems are not ready. Since there are the occasional times we are forced to deal with these operating systems, let’s take a look at the problems/issues, and what can be done (or not done) to help/fix them (in other words, let’s find the crutches we need to make them work with 3TB drives).
Several operating systems have switched to something called the GPT or GUID Partition Table as the layout mechanism on drives instead of the MBR (Master Boot Record) layout. It has been used on certain system BIOS for a while but it is part of Intel’s EFI (Extensible Firmware Interface) that is being promoted for new systems. GPT’s are 64-bit making our life a lot easier.
But the key issue is that your BIOS much support GPT to boot from a GPT enabled drive. In the consumer world this is a downside since most motherboards don’t have a BIOS that is GPT capable. This can affect all operating systems including Linux.
In addition to the BIOS, you also need to make sure your SATA controllers can also work with 64-bit LBA addressing. According to this article, Intel storage drives don’t support 64-bit LBA but running your ICH in Native IDE mode will work. According to the same site, the reason that Western Digital is including a SATA card with the new drives is to allow systems to use the 3TB drives since the card has a controller that is known to work with large drives. Be sure to check the specifications on your SATA controller to determine if it can work with GPT oriented drives.
But don’t feel too bad if your system doesn’t have an EFI BIOS or a BIOS that can handle GPT drives and doesn’t have a SATA controller that can handle large drives. These limitations just mean that you can’t boot from the 3TB drives. If your SATA controller can’t handle the larger drives you can still use them as a secondary or non-bootable drive that are typically used for non-OS data. The good news for the Linux world is that the SATA controller that comes with the drive can work with Linux and the 3TB drives, so we really don’t have any restrictions in using the 3TB drives as secondary drives. However, we are likely to be limited in using the drives as a boot drive – the same as everyone else because of the BIOS issue.
But there are likely to be factors affecting our choice of operating system. Figure 1, from Western Digital, shows the details on using the new 3TB drive as a boot drive or secondary drive with various OS’s.
Figure 1: OS compatibility chart for new Western Digital 3TB drive
Summary
We all feel the need for speed and large capacity storage and we don’t need to attend AA-esque meetings to admit it. Western Digital is the first company to offer a 3TB SATA drive that is focused on the consumer market. Their drive comes with a SATA controller to help existing systems take advantage of the 3TB of lovely storage capacity goodness if they meet other criteria (such as a SATA controller than can handle larger drives).
In addition to 3TB’s of capacity, this new drive also switches over to 4KB sectors. This switch offers increased capacity that is available for formatting compared to the more common 512 byte sectors. It can also help improve performance of the drive itself although some of the extra performance comes when the drive has bad sectors (a scenario we all want to avoid). In the meantime, the drive provides a 512-byte “compatibility” mode for older LBA oriented operating systems.
Linux is ready for 4KB drive sectors with 64-bit LBA addressing. Many SATA drivers are ready as well and Linux is even GPT ready, but the general consumer hardware world is not. The needed components should be ready in early 2011 (we’re hoping). The enterprise world may be a bit better off but you need to check on the motherboards in your servers. If they are using EFI you have cleared one hurdle. You can install Linux and boot from the drive but be sure you use the correct tools to handle GPT (i.e. “fdisk” is not a good idea). There are a number of sites that explain how to do this. If not, you can still use the 3TB drives if your SATA controller is ready, but only as secondary (i.e. non-bootable) storage (not a bad thing).
One last comment – since the appearance of the Western Digital 3TB drive, others have appeared. For example, the Hitachi Deskstar H3IK30003272SW has appeared. On newegg, as of the writing of this article, the drive retailed for $199.99. What is interesting about this drive is that it is a SATA III (6.0 Gbps) drive running at 7,200 rpm with a very large 64MB cache. I think that 2011 will see the combination of SATA III (6.0 Gbps), 3TB drives, and GPT tools becoming typical for systems especially since Intel is releasing new processors as is AMD. It’s going to be a great year Lewis.
Jeff Layton is an Enterprise Technologist for HPC at Dell. He can be found lounging around at a nearby Frys enjoying the coffee and waiting for sales (but never during working hours).
Comments on "3TB Drives are Here"
A very clear and informative article on high capacity drives. I like this better than long winded too much technical articles on tech sites.
I have heard that HD with capacity > 1 TB are very prone to failure. Is it a myth? If it is somehow true, can you please elaborate the technical reason? Thank you very much in advance.
well, bigger disks tend to have more platters and/or higher recording density. it’s pretty unclear whether either of those are correlated with increased failures, though. more platters affects bearings somewhat, the motor a little, and implies more heads, and more work for the head coil. higher recording density, holding everything else constant, is somewhat more sensitive to vibration, thermal issues.
so yeah: the most conservative approach is probably to stay a step back from the current highest-density, and to stick to 1-platter disks. IMO this is a bit silly, though, because disks are so cheap we really need to regard them more as consumables (plan <= 3 years use).
I got worried reading the first block of the article that it wouldn’t be useful to keep reading. Glad I kept at it, nice article, fills in some little holes that were missing.
10 bits in a Byte? 10 Octal, but most people don’t count in octal (or hex, for that matter) …
To tindallh:
Re: 10 bits in a Byte? You didn’t read the author’s next three words, which were “of ECC data”. When talking about real estate on the platter, you have to count the ECC bits.
For starters, great article including the fiction noire at the beginning.
The linked article says that “(hard) drive sizes are typically given in terms of total storage capacity, where 1 byte = 10 bits… This invisible (to the end-user) additional capacity is used to store positional information and for error correction code.” In other words one can estimate that for every N bytes of storage capacity visible to the user an additional 2N bits of total platter capacity are required.
However, when one is talking about a M byte sector with N extra bytes, then 1 byte is equal to 8 bits (as it is in the rest of the world). The 40 bits used for ECC is 5 bytes and not 4 bytes.
As luck would have it, one need not calculate bits. The article should read that a 512 byte sectors requires an additional 40 bytes for ECC and not 40 bits (a mistake which was also made in the linked article). Similarly, 512 byte sectors require an additional 320 bytes for ECC and 4K byte sectors require an additional 100 bytes for ECC. To note, 220 bytes of 4K is in the order of the quoted 5.5% additional capacity.
Good article, pointless intro, but great content
There’s an error regarding physical bit density on the platters
It areal density (as in area)
http://www.pcguide.com/ref/hdd/op/mediaDensity-c.html
Rather than aural density, aural refers to sound
There’s a lot of misleading stuff in this article…
It implies that LBA is the reason partitions are limited to 2.199TB. The fault lies with MBR partition tables, not LBA itself. GPT fixes this problem (mentioned far enough into the article that you disassociate the two).
It also doesn’t discuss the various types of Advanced Format drives, the most common of which expose 512-bytes to the OS. Little of Act II describes “real-world” hardware! Certainly not the 3 TB disk from Western Digital the author bought.
It also ignores the sector alignment problem, which is the most important issue/problem with 4K (and larger) sector disks. The sector alignment issues with Linux are the real reason Linux (the community and distributions, not the kernel) is not ready for 4 KB sector disks.
I expand on all of this on my blog: Comments on “3 TB disks are Here” from Linux Magazine. I intend on having more posts about working with GPT soon so you can mitigate all these problems.
There is a big whopper in this article “But the key issue is that your BIOS much support GPT to boot from a GPT enabled drive.” The assertion is untrue. I have a Dell Inspiron 1750 that I have successfully upgraded to GPT and there is no GPT support in the BIOS.
I’d suggest the author looks at http://www.rodsbooks.com/gdisk/whatsgpt.html and an article I wrote after I had upgraded my own laptop to GPT http://johnlewis.ie/converting-to-gpt-in-ubuntu/
In response to the previous commenter, your third point is on shaky ground. Linux is ready for 4k sectors (at least Ubuntu is, and I’m willing to bet most others too). Since 10.04 the partitioner in the installer defaults to a 2048 sector alignment. See https://wiki.ubuntu.com/LucidLynx/ReleaseNotes#Partition alignment changes may break some systems.
I also wrote an article on why partition/sector alignment isn’t nearly as big an issue as it once was http://johnlewis.ie/partition-alignment-largely-a-moot-point-now/
I don’t know who’s checking these articles over but this one needed to be better researched and more accurate. I hope you do better in future.
““But the key issue is that your BIOS much support GPT to boot from a GPT enabled drive.” The assertion is untrue.”
But your assertion is not quite true either. Yes, you can use GPT but you still, even in your article, use an MSDOS partition. As far as the BIOS is concerned you’ve booted off an MSDOS partition. It’s Grub that “bridges” to the GPT partitions.
On a system that truly supported GPT (Apples and Itaniums, for instance) you wouldn’t need the little MSDOS partition.
What assertion did I make other than you can use GPT with a machine with a standard BIOS?
It doesn’t matter whether you are using native “BIOS” support or Grub to do the work for you the fact is it can be done and so what I said is true.
And to correct you even further it isn’t called an MSDOS partition the correct terminology is “BIOS Boot Partition” type “EF02″, look it up.
http://grub.enbug.org/BIOS_Boot_Partition
Since this article focussed so much on Western Digital I would’ve liked to have seen some reference to the “Load cycle count” debacle. I want to buy a drive, install it, and just have it work. I don’t want to worry if my drive will prematurely wear out. I don’t want to have to fool around with SMART, WDIDLE, or to customize my OS for their disk. Do we have any assurance this won’t be an issue on the 3TB drives? (Not that I’m likely to buy WD after being burned on this.)
The problem has been around for while:
http://forum.synology.com/enu/viewtopic.php?f=124&t=11653
But is still an issue:
http://www.mail-archive.com/debian-user@lists.debian.org/msg587732.html
Their proposed solution is to run a DOS utility and/or have us modify our OS:
http://wdc.custhelp.com/app/answers/detail/a_id/5357
WD’s own admission of lack of Linux support:
http://wdc.custhelp.com/app/answers/detail/a_id/987/related/1
Couldn’t one just format the Drive with something like Palimpsest Disk Utility (a.k.a. gnome-disk-utility), and replace the MBR with the GUID Partition Table? This should correct the issue with the Drive Size limitation with MBR.
Grub cannot be embedded under a GPT partitioning scheme unless you use a BIOS boot partition (that I mentioned earlier).
A small amount of space is needed at the beginning of the drive before the first “real” partition for this.
GPT also wants to store a backup copy of the partition table at the end of the drive, again small amount of space after last partition required for this.
It is possible but not recommended to use block-lists to load Grub. Even where this works you still need to make space for the the backup partition table at the end of the drive.
However if something goes wrong it is possible to delete the GPT partition table and recreate a standard one without losing data, as long as you use the same start and end values for the partitions.
I doubt that Palimpsest deals with all that but feel free to try it for yourself and correct me if you are left with a machine that actually boots!
So to clarify I imagine Palimpsest will create the right sort of thing from scratch or “format” as you say, but if you have an existing OS you want to copy to a new large hard drive i.e. 2.2 TB and above, I think it’s easier to go through the conversion procedure I detailed again in the link provided earlier “Converting to GPT in Ubuntu”
You could definitely see your expertise within the work you write. The world hopes for even more passionate writers like you who are not afraid to mention how they believe. All the time go after your heart.
I don’t even know the way I stopped up here, but I believed this publish was great. I do not know who you are however certainly you are going to a well-known blogger in the event you are not already. Cheers!
Thank you for the good writeup. It in reality was once a entertainment account it. Look advanced to more added agreeable from you! By the way, how can we be in contact?
Wonderful points altogether, you simply received a new reader. What could you recommend about your put up that you simply made a few days ago? Any certain?