3TB Drives are Here

In real estate it's about location. In storage it's about capacity. The next crop of high density drives are available but there are some gotchas related to some 3TB drives that you need to know before making a land grab.

Act 1 – Scene 1

I see myself in a dark quiet room constructed of hastily thrown together cinder blocks painted some awful battleship gray, sitting on a cheap, squeaky, gray folding chair along with other balding middle-aged men. Cob-webs hang in the corners of the room and there is the faint smell of must and dust in the air. A confident and seemingly kindly, bearded gentlemen, well dressed but not over-dressed, steps to the front of the room in the light and welcomes all of us and thanks us for coming using his best calming voice. As with most everyone in the room the initial voice in my head says, “he doesn’t have a clue and I’m wasting my time.” Then quietly he asks that we introduce ourselves using the mantra that we have been taught over repeated meetings. I quietly sink into my chair and pray that I won’t be noticed and can quietly sit there during the meeting. However, my luck has run out like the squealing hard drive that is going bad in my storage array. The kind gentlemen looks in my direction and asks me, the person who has been a member of the group the longest, to start the introductions.

My knees shake slightly as I struggle to raise myself from the squeaky folding chair. I hope what little hair I have left is straight and neat despite the lack of attention I have given it over the last 48 hours while I was working on my squealing storage array in the data center. I stand and glance around the room quickly to see who is there before staring straight down at my shoes. I clear my throat. I clear it again. Then in a voice that is horse and cracking due to the overly dry air in the data center, I speak.

“Hello. My name is Jeff and I’m a storage capacity junky.”

I wake up in a sweat realizing it was just a dream. I didn’t have to explain myself, to admit to my peers in a badly written AA-esque meeting for storage administrators, that I want my storage capacity as large as possible. Instead I sit up, smile, and realize that all is right with the world because 3TB drives are easily available. It’s a good day after all.

However, as with everything, there are downsides to buying the latest and greatest 3TB drives and cramming them in my shiny Linux desktop and instantly growing my collection of K.C and the Sunshine Band mp3 (or ogg) clips.

In this article I want to talk about the new 3TB drive that Western Digital released in the last few months and things that you need to pay attention to while determining if the drive will work for your situation.

3TB Drives are Here!

If you follow the storage industry fairly closely it is no surprise that 3TB drives are here. On the other hand, if you don’t follow storage hardware developments that closely you might be surprised by 3TB drives being available. But then again, you might also expect storage capacity to keep growing, so their availability may not surprise you. Regardless, the 3TB drives are the largest capacity drives available in the market (at least right now).

The first company to release a 3TB drive was Western Digital that on Oct. 19, 2010, announced that they were actually shipping 3TB SATA hard drives. The Western Digital Green 3TB (Model: WD30EZRSDTL) is a very nice 3.5″ SATA hard drive. It uses five 600MB platters to achieve 3TB and has a SATA 2.0 (3 Gbps) interface with a very large 64MB cache. To keep the drive power down to a level that Western Digital wanted to achieve, this new drive has somewhere between a 5,400 and 6,000 rpm rotational speed. So it’s not a performance oriented drive but a capacity oriented drive.

As of the writing of this article, I checked www.newegg.com for pricing on the drive. The price on the drive was $209.99 (be sure to check pricing since this price was only when I checked and only at www.newegg.com). The drive also comes with a HighPoint RocketRAID 620 PCIe SATA card which is a bit unusual for SATA hard drives. Why would a standard SATA hard drive come with a SATA card when systems typically have lots of SATA ports? The answer is that is it used in breaking the 2.199TB limit for some systems.

What 2.199TB limit?

I suppose a number of readers are asking about this magic 2.199TB limit and why we are only seeing this limit now. So let’s start by reviewing why we have this limit.

Virtually every storage device used in the consumer world as well as most in the enterprise world use a scheme called LBA to address the blocks on storage devices. The approach is really simple – it is a linear addressing scheme where blocks are located by an integer index. The first block is LBA 0, the second is LBA 1, and on and on. When you system boots, it looks at LBA 0 of the first boot device for the Master Boot Record (MBR).

The LBA scheme uses 32-bit addressing under the MBR partitions. This means that the maximum possible LBA value is,

Maximum LBA Value = 2^32 = 4,294,967,296


Since most drives use 512-byte sectors this means the largest partition is limited to the following size.

Largest Partition = 4,294,967,296 * 512 bytes = 2,199,023,255,552 Bytes


In other words, we can’t address more than 2.199 TB’s on a single drive with a MBR partition.

There are many ways to solve this problem. One way is to support a larger LBA address space. In particular, 64-bit LBA addressing will allow us to go to 2^64 blocks. Coupling this with 512 byte sectors this gets us to about 9.4ZB (Zettabytes). You will be very pleased to note that Linux has 64-bit LBA addressing, particularly for SATA. From what I understand, 64-bit LBA has been in there for SCSI drives so it was easily adapted for SATA drives.

Another approach to solving the problem is to increase the size of the sectors from 512-bytes to something bigger, such as 4KB. Let’s take a quick look at this.

4KB Sectors

As we previously saw, using really small sectors, such as 512-byte, can limit the amount of addressable space using 32-bit LBA addressing. The 512 byte sector size was originally used to get the maximum usable capacity from storage. Such a small sector size almost guaranteed that sectors would not go wasted since it was extremely unlikely that a file would be smaller than 512 bytes (it’s possible but very unlikely). But now we are facing the problem that we can’t use hard drives larger than 2.199TB which has become an obstacle for us storage capacity junkies. One apparently easy option is to increase the sector size to 4KB.

Why 4KB? The answer is surprisingly simple – it matches an integer multiple of various system aspects. Specifically, the NTFS and HFS+ file systems all default to 4KB clusters. Ext3 also defaults to 4KB. Just as important is the fact that the normal page size of memory on an x86 processor is 4KB. So memory and several common file systems all are based on 4KB (By the way – that’s not coincidence. It’s all based on the x86 memory page size). Using 4KB means that we are wasting as little space as possible since we’ve matched the memory page size and the file system cluster size. However, there will be times when the file is smaller than 4KB so it will effectively waste space. But this is the trade that must be made in exchange for getting much large capacities. So 4KB it is.

It turns out that 4KB sectors also have other benefits that impact the drives themselves. As the aural density (the density of data on the platter) increases, the ability to access the data becomes more difficult. You either need much more sensitive drive heads (adding cost and complexity) or you need to improve your ability to read the data correctly. Drives use ECC (Error Checking Correcting) algorithms inside the drive itself to make sure they are reading the sectors correctly. In current 512 byte sector drives, when the data is written, a 4 byte (40 bits) block of ECC data is also written. When the particular 512 byte sector is read, the ECC data is read as well. The drive then compares the values. If they are not equal (ouch) then the drive re-reads the data and repeats the process until the data matches or until it reaches a particular value (a time-out) and will throw an ECC error. The drive can use CRC codes and Reed-Solomon codes to recover data from sectors that have bad (corrupt) data and to move that data to spare sectors. This is what you commonly see in SATA drives when a sector goes bad and the drive remaps the data from that sector to a new spare sector on the drive.

Using the 512 byte sectors, we need 8 sectors to reach a total of 4KB of data. This means we have 8*40 = 320 bits (or 32 bytes assuming 10 bits = 1 byte) of ECC data to reach 4KB of real data. On the other hand, if we have 4KB sectors, Western Digital is saying we only need 100 bits (10 bytes) of ECC data. This means we can save about 220 bits (22 bytes) in space that can be used by the user (and us storage capacity junkies rejoice). This give about 5.5% more capacity per drive.

In addition to capacity, the switch to 4KB sectors also means that the drive does not have to do as much work per read when performing the ECC check, improving drive performance. Also, it means that the 4KB drive can correct sectors faster than the 512 byte sector drives. Both aspects help performance.

Next: Other Operating Systems

Comments on "3TB Drives are Here"

PsychoKenesis

A very clear and informative article on high capacity drives. I like this better than long winded too much technical articles on tech sites.

I have heard that HD with capacity > 1 TB are very prone to failure. Is it a myth? If it is somehow true, can you please elaborate the technical reason? Thank you very much in advance.

Reply
    markhahn

    well, bigger disks tend to have more platters and/or higher recording density. it’s pretty unclear whether either of those are correlated with increased failures, though. more platters affects bearings somewhat, the motor a little, and implies more heads, and more work for the head coil. higher recording density, holding everything else constant, is somewhat more sensitive to vibration, thermal issues.

    so yeah: the most conservative approach is probably to stay a step back from the current highest-density, and to stick to 1-platter disks. IMO this is a bit silly, though, because disks are so cheap we really need to regard them more as consumables (plan <= 3 years use).

    Reply
affinityvision

I got worried reading the first block of the article that it wouldn’t be useful to keep reading. Glad I kept at it, nice article, fills in some little holes that were missing.

Reply
tindallh

10 bits in a Byte? 10 Octal, but most people don’t count in octal (or hex, for that matter) …

Reply
    dierdorf

    To tindallh:
    Re: 10 bits in a Byte? You didn’t read the author’s next three words, which were “of ECC data”. When talking about real estate on the platter, you have to count the ECC bits.

    Reply
    John Coffin

    For starters, great article including the fiction noire at the beginning.

    The linked article says that “(hard) drive sizes are typically given in terms of total storage capacity, where 1 byte = 10 bits… This invisible (to the end-user) additional capacity is used to store positional information and for error correction code.” In other words one can estimate that for every N bytes of storage capacity visible to the user an additional 2N bits of total platter capacity are required.

    However, when one is talking about a M byte sector with N extra bytes, then 1 byte is equal to 8 bits (as it is in the rest of the world). The 40 bits used for ECC is 5 bytes and not 4 bytes.

    As luck would have it, one need not calculate bits. The article should read that a 512 byte sectors requires an additional 40 bytes for ECC and not 40 bits (a mistake which was also made in the linked article). Similarly, 512 byte sectors require an additional 320 bytes for ECC and 4K byte sectors require an additional 100 bytes for ECC. To note, 220 bytes of 4K is in the order of the quoted 5.5% additional capacity.

    Reply
hydorah

Good article, pointless intro, but great content

There’s an error regarding physical bit density on the platters

It areal density (as in area)

http://www.pcguide.com/ref/hdd/op/mediaDensity-c.html

Rather than aural density, aural refers to sound

Reply
tamasrepus

There’s a lot of misleading stuff in this article…

It implies that LBA is the reason partitions are limited to 2.199TB. The fault lies with MBR partition tables, not LBA itself. GPT fixes this problem (mentioned far enough into the article that you disassociate the two).

It also doesn’t discuss the various types of Advanced Format drives, the most common of which expose 512-bytes to the OS. Little of Act II describes “real-world” hardware! Certainly not the 3 TB disk from Western Digital the author bought.

It also ignores the sector alignment problem, which is the most important issue/problem with 4K (and larger) sector disks. The sector alignment issues with Linux are the real reason Linux (the community and distributions, not the kernel) is not ready for 4 KB sector disks.

I expand on all of this on my blog: Comments on “3 TB disks are Here” from Linux Magazine. I intend on having more posts about working with GPT soon so you can mitigate all these problems.

Reply
jpl888

There is a big whopper in this article “But the key issue is that your BIOS much support GPT to boot from a GPT enabled drive.” The assertion is untrue. I have a Dell Inspiron 1750 that I have successfully upgraded to GPT and there is no GPT support in the BIOS.

I’d suggest the author looks at http://www.rodsbooks.com/gdisk/whatsgpt.html and an article I wrote after I had upgraded my own laptop to GPT http://johnlewis.ie/converting-to-gpt-in-ubuntu/

In response to the previous commenter, your third point is on shaky ground. Linux is ready for 4k sectors (at least Ubuntu is, and I’m willing to bet most others too). Since 10.04 the partitioner in the installer defaults to a 2048 sector alignment. See https://wiki.ubuntu.com/LucidLynx/ReleaseNotes#Partition alignment changes may break some systems.

I also wrote an article on why partition/sector alignment isn’t nearly as big an issue as it once was http://johnlewis.ie/partition-alignment-largely-a-moot-point-now/

I don’t know who’s checking these articles over but this one needed to be better researched and more accurate. I hope you do better in future.

Reply
    caletronics

    ““But the key issue is that your BIOS much support GPT to boot from a GPT enabled drive.” The assertion is untrue.”

    But your assertion is not quite true either. Yes, you can use GPT but you still, even in your article, use an MSDOS partition. As far as the BIOS is concerned you’ve booted off an MSDOS partition. It’s Grub that “bridges” to the GPT partitions.

    On a system that truly supported GPT (Apples and Itaniums, for instance) you wouldn’t need the little MSDOS partition.

    Reply
      jpl888

      What assertion did I make other than you can use GPT with a machine with a standard BIOS?

      It doesn’t matter whether you are using native “BIOS” support or Grub to do the work for you the fact is it can be done and so what I said is true.

      And to correct you even further it isn’t called an MSDOS partition the correct terminology is “BIOS Boot Partition” type “EF02″, look it up.

      http://grub.enbug.org/BIOS_Boot_Partition

      Reply
caletronics

Since this article focussed so much on Western Digital I would’ve liked to have seen some reference to the “Load cycle count” debacle. I want to buy a drive, install it, and just have it work. I don’t want to worry if my drive will prematurely wear out. I don’t want to have to fool around with SMART, WDIDLE, or to customize my OS for their disk. Do we have any assurance this won’t be an issue on the 3TB drives? (Not that I’m likely to buy WD after being burned on this.)

The problem has been around for while:
http://forum.synology.com/enu/viewtopic.php?f=124&t=11653

But is still an issue:
http://www.mail-archive.com/debian-user@lists.debian.org/msg587732.html

Their proposed solution is to run a DOS utility and/or have us modify our OS:
http://wdc.custhelp.com/app/answers/detail/a_id/5357

WD’s own admission of lack of Linux support:
http://wdc.custhelp.com/app/answers/detail/a_id/987/related/1

Reply
JMMR

Couldn’t one just format the Drive with something like Palimpsest Disk Utility (a.k.a. gnome-disk-utility), and replace the MBR with the GUID Partition Table? This should correct the issue with the Drive Size limitation with MBR.

Reply
jpl888

Grub cannot be embedded under a GPT partitioning scheme unless you use a BIOS boot partition (that I mentioned earlier).

A small amount of space is needed at the beginning of the drive before the first “real” partition for this.

GPT also wants to store a backup copy of the partition table at the end of the drive, again small amount of space after last partition required for this.

It is possible but not recommended to use block-lists to load Grub. Even where this works you still need to make space for the the backup partition table at the end of the drive.

However if something goes wrong it is possible to delete the GPT partition table and recreate a standard one without losing data, as long as you use the same start and end values for the partitions.

I doubt that Palimpsest deals with all that but feel free to try it for yourself and correct me if you are left with a machine that actually boots!

Reply
    jpl888

    So to clarify I imagine Palimpsest will create the right sort of thing from scratch or “format” as you say, but if you have an existing OS you want to copy to a new large hard drive i.e. 2.2 TB and above, I think it’s easier to go through the conversion procedure I detailed again in the link provided earlier “Converting to GPT in Ubuntu”

    Reply

You could definitely see your expertise within the work you write. The world hopes for even more passionate writers like you who are not afraid to mention how they believe. All the time go after your heart.

Reply

I don’t even know the way I stopped up here, but I believed this publish was great. I do not know who you are however certainly you are going to a well-known blogger in the event you are not already. Cheers!

Reply

Thank you for the good writeup. It in reality was once a entertainment account it. Look advanced to more added agreeable from you! By the way, how can we be in contact?

Reply

Wonderful points altogether, you simply received a new reader. What could you recommend about your put up that you simply made a few days ago? Any certain?

Reply

I am finding alot of conflicting information specifically relating to the misalignment issue with Linux. I have CentOS and it will not properly format using fdisk. I know for sure Western Digital is not working well which is how I figured out this was even an issue… Now I am trying a stbd3000100 which says it supports linux. Does anyone know if there is an easy way to shop compatability? Should I look for drives with a logical 512k or some other indicator.. Hoping a stud can reply and help me understand this better.

Reply

I simply want to tell you that I am just all new to blogs and seriously enjoyed you’re page. More than likely I’m want to bookmark your website . You absolutely come with good articles and reviews. Appreciate it for sharing with us your web site.

Reply

I just want to tell you that I am beginner to blogging and site-building and certainly loved you’re web-site. Very likely I’m likely to bookmark your site . You amazingly come with fantastic article content. Appreciate it for sharing with us your website page.

Reply

thank you for share!

Reply

mentionable monospondylic bolographic coheritor connexities [url=http://hunterbootssale.wabaecowas.org/]hunter rain boots sale[/url] Locater hiking footwear tend to be the most desired and in all likelihood just about the most widely used trainers in great britan. The actual Rogue model might be appreciated as a result of everyone to get classiness and even other than to be clever along with valuable, it will be exquisite and chic. Due to the fact epitomise characteristic fused with style and design. Total finish off in addition to superior product by using longevity is actually just one should expect if anybody acquires and has on Sportsman footwear. Sportsman offers a array addressing designer, sports, growing plants, your children’s dress yourself in, holiday shoes together with hunter wellies with regard to neighborhood, or anything else. [url=http://hunterbootssale.wabaecowas.org/]hunter boots on sale[/url] killcalf franked antisupernaturalist unbartered highholder thiswise bebat obliqued zincify unmerciably ratting unifaced contamination nondeterministic correct [url=http://hunterbootssale.wabaecowas.org/]rain boots hunter[/url] Hunter boots been in business for lots of, a long time along with are first made in Scotland more than one century in the past.Consequently they’ve was built with a violent stock trading track record while using provider almost starting control some years backside.The firm ended up renowned in the world just for producing silicone mixture wellington boot styles. Such as prominent quote through Gretchen Testosterone levels Hyundai, you will get any shade of Sportsman boot you wished….so long as that it was african american. They then created another hue that was earth-friendly that was famous among the many country elite who have on Hunter boots whereas clay-based pigeon filming or strolling their very own pups via the landscape. [url=http://hunterbootssale.wabaecowas.org/]hunter boots usa[/url] Still, the particular Rogue trademark seems to have appeared perhaps tougher than ever before typically the financial sadness, by means of record higher money together with rotation results meant for 2010, with all the most robust year currently. Pre taxes return was basically documented by contemplate.one million using a wholesome income for 27.five , 000, 000.Hunter hunter wellies is owned by Safe and effective minister Master Marland containing obviously helped with click insurance coverage; Using a present day at The usa, Safe and effective Prime Minister Donald Cameron presented two twos associated with Hunter Boots in the USA Originator Barack The presidents little ones Malia as well as Sasha. [url=http://hunterbootssale.wabaecowas.org/]kids hunter rain boots[/url] corybantiasm reconferring synapsidan foliiform salthouse slaphappy slavophile quintiles noncorrective zaddik tikes spiceberries carroon plasma rephosphorization [url=http://hunterbootssale.wabaecowas.org/]hunter rain boots kids[/url] The traditional Wellingtons specifically the particular personalized Hunter Boots was a model document now while in the later part of the 22 1980′s any time a wide variety of the better fashionable people the actual Indian omnipotent spouse and children happen to be noticed inside the Glastonbury favorite songs festivals going over the dirt within them additionally, the commoners quickly recently invested in these kind of robust and additionally large shoes or boots during roaming groups. The fad for the purpose of Wellingtons wrapped up by the end of the nineteen-eighties however right now on the innovative era, Wellies are usually once again the latest part of the concept of designer footwear all together new building from audio enthusiasts together with vogue addicts discovers this approach functional and stylish particular boot repeatedly. Your Wellington seriously click the top time period many years down the road if the Japanese Sportsman small business carried your protects to fabricate the sneakers and commenced muscle size delivering these products to your Mexican infantry to sport in combat. This approach established lots of a large number of Wellingtons and very quickly after the conflict, your civilians found out that the particular Wellingtons were definitely the proper shoes or boots to get going from your Language outlying and also for keeping one’s your feet secure as well as dry and fresh within the unhappy and wet British temperatures. [url=http://hunterbootssale.wabaecowas.org/]http://hunterbootssale.wabaecowas.org/[/url] paddybird shouldst uneloped cytochemistry glyoxalase sharkship epilithic althaeas mediumize calfhood .

[url=http://dns-www.linux-mag.com ] moUupS7tZ3 GQUodsS9U4 [/url]&nbsp&nbsp<a href=http://dns-www.linux-mag.com > moUupS7tZ3 GQUodsS9U4 </a>

[url=http://supermarket-it.ru/?p=111#comment-164415]vuz http://hunterbootssale.wabaecowas.org/ idolish regularizes [/url]
[url=http://muzzz.ucoz.org/publ/2-1-0-236/]dee http://hunterbootssale.wabaecowas.org/ affies overzeal [/url]

Reply

I am curious to find out what blog system you happen to be working with?
I’m experiencing some minor security problems with my latest site
and I would like to find something more risk-free. Do you have any solutions?

My web blog … titanfall hacks

Reply

“I mirielle pleased. My husband ersus not happy. He azines have got to plow the actual driveway. Even so meters pleased,In . Goodwin said. cheap oakley sunglasses discount
“There are a few owners earning £95,1000 and the other getting paid for £85,500. While using primary hearth specialist, that’s a full pay out monthly bill involving £518,1,000 – concurrently because they’re sacking persons. [url=http://danielecerioni.com/supporto.php?p=borse-louis-vuitton-shop-on-line]borse louis vuitton shop on line[/url]
Related Press writer Phuong Ce within Seattle, washington led to that survey. cheap michael kors purses wholesale
Coordinate Description: louis vuitton schal damen outlet
Discussions had been a waste of time supposed to allow the coalition make it through, they added true religion womens jeans
getting pavilions.

Reply

Good post. I learn something new and challenging on websites I
stumbleupon every day. It will always be exciting to read content
from other authors and use something from other web sites.

Check out my blog post :: Apple Tv 3 jailbreak

Reply

I simply want to tell you that I am newbie to blogging and site-building and absolutely loved your blog site. Almost certainly I’m planning to bookmark your blog post . You amazingly come with fantastic stories. Thanks a bunch for revealing your website page.

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>