These are the type of articles and responses I really like. Thanks and I look forward to more.

That said, is bit rot, as described by the author, even a real world problem any more? I was under the impression that disks were so advanced now that even a single rotted bit can be detected upon a read and marked as such, telling the operating system that the drive had a “read error” at which point you are notified of the defect and can take action. So all it takes is that you just run a disk scan or defrag or do something else that causes a read across your entire drive and you will know. It doesn’t stop the bit from rotting, of course, but it is far from a silent killer.

A UBER of 1 in 1E14 is a UBER of 1E-14, not 1E14.

If the UBER for one drive is 1E-14 then the UBER for two drives is still 1E-14. It doesn’t matter how many disks you have or how big they are – you still expect to get one unrecoverable bit error for every 1E14 bits read from it/them. The bit error rate is not in terms of how much data you have stored, but how much data you read.

The UBER is an average error rate, not a deterministic process. That is, you expect to get one bit error for every 1E14 bits you read from the disk. If you read 1E15 bits from the disk, you expect ten bit errors. It might be none, it might be ten, it might be every single one of those 1E15 bits that are wrong, but if you do it over and over again you expect the average to be one out of every 1E14 bits.

To calculate the probability of k bit errors after reading N bits you have to use the binomial distribution. GNU R is the only thing I can find that can calculate it at these sorts of probabilities and number of trials. It shows that after reading 1E14 bits, the probability of at least one bit error is 63%, at lesat two is 26%, at least three is 8%, at least four is 2% and so on. Alternatively, the probability of no errors after reading 1E14 bits is 37%, not 0.

No matter how much data you read, the probability of having encountered and error never reaches 1.

I have one comment on the URE issue.
I don’t claim to be an expert on bit rot, URE, or computer hardware. I do, however, know a little bit about BER (bit error rate) as it relates to my line of work.
An error rate of 1 in 10^14 is usually an average!
To interpret it, we imagine having a very large number of bits and counting all occurrences of URE in them. The error rate is then the number of URE divided by the (large number) of bits we’ve read. The key word here is large! For this measure to have any meaning, you must count a large number of bits.
What I want to say is, in theory, if your drive consists of 3 bits (bear with me), and the manufacturer has reported an error rate of 1 in 10^14, there’s nothing to prevent all 3 bits from having a URE. It’s very unlikely but possible.
The complete opposite is also possible: if you have a 12 TB hard drive and you read it bit by bit, you may find absolutely no URE’s! If you’ve read all of the 12 TB except one bit and found no URE’s does NOT make you more likely to find the last bit being in error. What makes us usually think this way is a cognitive bias known as the Gambler’s fallacy.

I’m sorry if this seems like nitpicking or arrogant presumption on my part. I’m not undermining your expertise in any way. Thank you for the great article!

Take a look on PAR2 (GNU) or ICE ECC (Windows).

In fact I believe the chance of an error would never hit one, since the odds of an error would be (1E14)^(infinity) or 1E(infinity), at which point the odds of an error would be as close to zero as to be a non-event for all intents and purposes.

This is exactly what google does more or less with their data- couple of years ago, the idea was to take a “majority vote” among 3, 5 and so on replicas.

PS: running scripts as “root” is NEVER a good ideea – you may screw up all the permissions on the files you act and they may end up owned by root and therefore “invisible/untouchables” to all the other developers.