On-line Backups: Flexible Enough for Home & the Office

Backups are a technology or process that everyone -- everyone! -- needs to consider. This article looks at some on-line backup options for Linux that can apply to the spectrum of home to enterprise-class users.

Last week’s column featured a discussion about using Linux software RAID (md) to provide additional data protection in addition to backups. Md can be used for RAID-1 (mirroring) or RAID-5 or RAID-6 to give extra data protection in the event of the loss of a drive. But the primary data protection scheme for almost all situations is a backup. This is true for home systems or laptops, up to very large server farms (there are some questions about doing backups on very large size storage systems but that’s a subject for another day).

This article focuses a bit more on backups for home users and Small and Medium Business (SMB). In particular, this article looks at the concept of on-line backups. That is, backups to a storage farm on the Internet. However, this discussion is not limited to home users and SMB’s but can be applied to most any company or storage need.

Taxonomy of Backups

It is well beyond the scope of this article to discuss the fundamentals of system backups, but for the sake of consistency, let’s do a quick review of backups. A backup is a simple concept of making a copy of data that can then be used to either restore a complete file system in the event that it crashes, or to restore individual files in the event that files are erased (accidentally or otherwise). The complexity behind backups comes from the details of how this process occurs and how it it managed.

While Wikipedia is not the most authoritative source of information, the basic concepts of backups are explained very well and serve as a good starting point for explaining the details of backups including management. It breaks down backups into a easily understood hierarchy:

  • It starts by discussing “Data Repository Models” which are a description of “how” the data is stored. For example,

    • Unstructured
    • Full + incremental (arguably the most common)
    • Continuous data protection

  • A discussion of “Storage Media” for backups

    • Magnetic tape
    • Floppy drives (for the older crowd)
    • Hard drives
    • Optical drives
    • Solid-state drives
    • Remote (on-line) storage

  • Techniques for managing the data repository

    • On-line
    • Off-line
    • Near-line
    • Off-site, off-line vault
    • Backup site of disaster recovery (DR) site

  • Then there is a discussion about selecting and manipulating data that is to be backed up including how to handle live data and metadata

    • Copying files
    • Partial files
    • Filesystem dump
    • Identification of changes
    • Versioning file system

  • There is a discussion about the manipulation of data and dataset optimization (this is where the concept of de-duplication comes in). It discusses the concepts of:

    • Compression
    • De-duplication
    • Encryption
    • Multi-plexing
    • Refactoring
    • Staging

  • Finally there is a section that talks about managing the backup process. It discusses such topics as:

    • Objectives

      • Recovery Point Objective (RPO)
      • Recovery Time Objective (RTO)
      • Data security

    • Limitations (what limitations are there in the overall process):

      • Backup window
      • Performance impact
      • Costs of hardware, software, labor
      • Network Bandwidth

    • Implementation

      • Scheduling
      • Authentication
      • Chain of trust

    • Measuring the process (making sure the backup scheme worked)

      • Backup validation
      • Reporting
      • Logging
      • Validation
      • Monitored Backup

The discussion is very thorough and logical. It is well worth reading even if you are doing backups today.

This article will focus on the backup media from the above taxonomy of backups. In particular it will focus on on-line (remote) backups.

On-line Storage Concepts

The idea of storing backups on-line is really not a difficult one. The fundamental idea is to store data, either full backups, or incremental data, on a remote storage server that is located somewhere on the Internet. You can then use this stored data in whatever manner you see fit. For example, it could be used for restoring systems in the event of a storage failure, or restoring individual files that have been erased. As long as the system requiring the data has access to the Internet, the data can be retrieved.

One additional concept for on-line backups is sharing files. One user can copy their files to the on-line backup and then other users, sharing the same backup, can download the files as needed. You can also extend this concept to something called synchronization. That is, making sure you have the latest files on several of your systems (i.e. the same data is on all of the systems). For example, you can use an on-line backup to upload your important files from your desktop and then download them to your laptop when you are on the road.

Since storage is not free, on-line backups are commercial enterprises with varying cost structures. There are really two main types of backup services

  • Services that use standard Linux tools such as rsync

  • Services that use a custom tool, sometimes web-based

Which approach is better depends upon you, your preferences, your processes, and the cost structure you are comfortable with.

On-line Backup Services Using Standard Linux Tools

The most common tool used for this type of on-line backup is rsync. Rsync is a file copying tool that can be used on local hosts or across remote hosts using a remote shell or a rsync daemon. A key feature of rsync is that it can send just the differences between two files rather than the entire file, saving bandwidth and time. This makes rsync a very popular tool for backups.


One service that is available that uses rsync is “rsync to Amazon’s S3″ by S3rsync.com. Amazon has developed a “cloud” storage product called Amazon S3 (Simple Storage Service). S3 is used by a number of companies because of the relatively low cost nature and also because it can be expanded at any time without having to buy hardware. The cost structure of S3 is based on three aspects:

  • Capacity (how much are you storing)
  • Data transfer (how much data is transferred in and out)
  • Requests (getting and putting data)

The rates are also based on monthly usage patterns.

While S3 sounds really great there a couple of caveats to it’s usage. First, S3 does not have a guarantee about not losing data but given the size and scope of S3, losing data is not likely. In addition, to access S3 you need to use Amazon’s tools so they are not easily integrated into existing processes and tools. Moreover, the S3 storage protocol allows you to only ‘get” or “put” data so you cannot modify a file that is stored in S3. If you modify even 1 bit in the file, you have to download the entire file, modify it, and upload the entire file again, leading to extra cost and burning up bandwidth between S3 and your servers.

S3rsync.com has developed a simple way to integrate rsync with S3. The basic concept is to use rsync to S3rsync.com’s servers which then use S3 for their storage. What R3sync.com has done is to put their rsync servers inside the Amazon EC2 facility to allow easy access to S3. All you need is rsync (comes with every Linux distribution) and ssh. In addition there is currently a 10GB limit on buckets (see the S3 site for more information on buckets).

As of the writing of this article, S3rsync.com charges $0.05 per hour in addition to the standard S3 charges.


Another on-line backup service that uses rsync is rsync.net. It is a very popular service, primarily because of it’s support (i.e. engineers man the phones for the most part). Currently, rsync.net supports:

  • rsync/scp/sftp

  • unison

  • rdiff-backup

  • It also supports certain commands using ssh

  • You can mount the file system using sshfs

  • An encryption tool called duplicity that uses rsync but also performs GPG encryption in the process

  • Remote subversion repositories
  • Remote data dumps of MySQL and PostgreSQL databases

Rsync.net also does not have a traffic charge so you can move as much data into and out of the file system as you need. Also, rsync.net allows multiple users to share one account (great for work groups and/or file synchronization).

Pricing for rsync.net as of this writing starts at $1.20/GB per month up to about 50GB ($60/month). Above 50GB there is a discount. If you want to have redundant copies of the data in two different locations, the price is $2.10/GB per month for up to 50GB’s ($110/month).

On-line Backup Services Using Custom Tools

The second class of backup services uses a custom tool or is web based. Some of them may also take a different approach in that the files that are to be backed up have to be in the same directory (this approach works better for work groups and file synchronization). A key feature of this class is that many of them offer free backups up to a certain limit (not a bad way to test the backup services).

Comments on "On-line Backups: Flexible Enough for Home & the Office"

Fascinating blog! Is your theme custom made or did you download it from
somewhere? A design like yours with a few simple adjustements would really make my blog shine.

Please let me know where you got your design.

I do not even know how I ended up here, but I thought this post was good.
I do not know who you are but certainly you’re going to a famous blogger if you are not already ;) Cheers!

Sweet blog! I found it while browsing on Yahoo News. Do you have any suggestions on how to
get listed in Yahoo News? I’ve been trying for a while but I never seem to get there!

Appreciate it

I could not refrain from commenting. Well written!

You really make it seem so easy with your
presentation but I find this matter to be actually something which I think I would never understand.
It seems too complicated and very broad for me. I am looking forward for your next post,
I’ll try to get the hang of it!

Hello! I could have sworn I’ve been to this web site before but after browsing through
many of the articles I realized it’s new to me. Anyhow, I’m certainly pleased I found it and I’ll be
bookmarking it and checking back often!

Unquestionably believe that that you said. Your favourite reason appeared to
be at the web the simplest thing to remember of.
I say to you, I certainly get annoyed whilst other
people consider concerns that they just do not realize about.
You managed to hit the nail upon the top as smartly as defined out the
entire thing with no need side effect ,
people can take a signal. Will probably be back to get more.


I reallyy like your blog.. very nice colors & theme. Did you design this website yoourself or
did you hire someone to do it for you? Plz respond as I’m looking to construct my own bloog and would like to find
oout where u got this from. thanks a lot

my web page – earning free golden eagles

I was wondering if you ever thought of changing
the layout of your blog? Its very well written; I love whhat youve got to say.

But maybe you could a little more in the
way of content soo people could connect wiuth iit better.
Yuve got an awful lot of texzt for only having one or 2 pictures.
Maybe you could space it out better?

My paage – earning free golden eagles

They’re a monopoly, yup act software download (http://radio.allmusicsearch.com/top/index.php?a=stats&u=violetchapdelain)
The powerbook computer itself is less expensive
so it might be an alternative if you’re building a
system yourself microsoft office software Additionally you can add a backlink to
website is in your ebook’s start menu directory 3d animation software download (digitalvideoconcept.com) On the other hand, if you’d like
a machine that can do everything, well then, your best served by a P belkin easy transfer cable
software download (weeventos.com.br) Around the Windows side install, also install FS-Driver which just might help you access
Ext2 and Ext3 formatted disks Free Dj mixing Software download The story book becomes a den or tree house where
children can go inside, shut the door, and appreciate download
video Web Camera Software Download – http://Www.Martaceramica.Com,;

It reϲords all the trаnsaϲtiߋns taкіng
placе іn a comρany
ѕo tɦаt уօu can alѡаʏѕ һaѵе
a cɦeϲҝ օn tɦe lоѕѕеѕ іf гᥱգսігᥱⅾ and
tⲟ ɡenerɑⅼlʏ aϲcеρteԀ ɑсϲօսntіng
tһе ƅаsіc framеԝοгқ οf ɑсcоuntіng sҺоսⅼⅾ
aⅼԝɑүѕ inclսɗе financᥱ ɑnd ɑϲϲoսntіng
tһеѕе cuѕtomегѕ іn ɦοա tο ɦаndlе tҺᥱіr taх affaіrѕ іn tҺе
fᥙtᥙге cаn ѕaνe tҺеm
mаny tҺоuѕаndѕ ⲟf dօⅼⅼaгѕ.
Ι dеϲіԁеⅾ
tߋ ᥙѕе tҺeiг OΙᏟ ѕеrνіϲе tօ ⅼоաег mʏ taҳ ƅіll.

ᒪⲟоκ ɑt mʏ
ᴡᥱbѕіtᥱ: Boca Raton CPA jobs California

Thus, the damage is not really sent a person but to others as well avira Free Sms software download – Vizatula.ru, antivirus software download (Kandis) The laptops
are intrinsically very fashioned to do multi-tasking siemens software download (http://www.tweetitis.com)
Although Lisa decided not to continue further but Apple was determined to do better with Mac computers
free interior design software download Ideally, your services
and products seem enough to accomplish this task microsoft office free download Plus,
they open the door for you be that can improvise linksys
software Download (piperdm.com) Futura and other alike hair systems can extend to 400 degrees even though many women do
not test the limits within the dj software free download full version

its form, especially if you’d like compose DVDs Neue bucher its form, especially if you’d like
compose DVDs free library buch downloads its form, especially if you’d like compose DVDs
kostenloses online romantik bucher zum herunterladen
its form, especially if you’d like compose DVDs ebook downloads kostenlos its form, especially
if you’d like compose DVDs download books to read its
form, especially if you’d like compose DVDs free book download-websites

My homepage: free ebooks zum herunterladen

sac longchamp série limitée
kate moss très rare état neuf Sac Longchamp Solde jeux chiffres
Longchamp de chasseurs de monstres
boutique longchamp
sac longchamp solde
sac longchamp homme

sac à mains longchamp
planète toile et cuir noir taille s ! petit modèle Sac a Main Longchamp plage
Longchamp fourre-tout poudre de bain chanel
sac seau longchamp
bracelet longchamp
prix sac longchamp

♥♥♥♥ longchamp: sac à main tout cuir blanc gamme roseau ♥♥♥
Sac Longchamp Neuf Pas Cher sac Longchamp hobo surdimensionné ugg boots
de sortie
longchamp solde
soldes sac longchamp
sac a main longchamp pas cher

It’s hard to come by well-informed people about this topic, but you sound like you know what you’re talking about! Thanks

I’ll gear this review to 2 types of people: current Zune owners who are considering an upgrade, and people trying to decide between a Zune and an iPod. (There are other players worth considering out there, like the Sony Walkman X, but I hope this gives you enough info to make an informed decision of the Zune vs players other than the iPod line as well.)

It’s hard to find well-informed people in this
particular topic, but you seem like you know what you’re talking about!

Good post. I learn something totally new and challenging
on sites I stumbleupon on a daily basis. It will always be useful to read content
from other writers and practice a little something from other sites.

Hello there! I know this is kind of off topic but I was wondering if you knew where I could get
a captcha plugin for my comment form? I’m using the same blog platform as yours and I’m having difficulty
finding one? Thanks a lot!

Very shortly this site will be famous among all blogging
and site-building people, due to it’s pleasant articles

I know this website gives quality depending articles and
additional stuff, is there any other web site which offers such stuff in quality?

bookmarked!!, I love your blog!

Just desire to say your article is as astonishing.
The clarity on your publish is simply spectacular and i
could suppose you’re knowledgeable on this subject.
Well along with your permission let me to take
hold of your RSS feed to stay updated with impending post.
Thank you 1,000,000 and please carry on the enjoyable work.

I for all time emailed this web site post page to all my associates,
since if like to read it after that my links will too.

Definitely believe that which you said. Your favorite reason seemed to
be on the internet the simplest thing to consider of. I say to you,
I definitely get annoyed even as other people consider concerns that they just
don’t understand about. You managed to hit the nail upon the top as neatly as outlined out the whole thing without having side effect , people could take
a signal. Will likely be again to get more. Thanks

Good article. I certainly appreciate this site. Keep writing!

It’s going to be ending of mine day, but before end I am reading this enormous article to improve my experience.

I really like what you guys are up too. Such clever work and coverage!

Keep up the awesome works guys I’ve you guys to my personal blogroll.

This paragraph will help the internet users for creating new webpage or even a blog from start to end.

I needed to thank you for this great read!! I absolutely loved every little bit of
it. I have got you bookmarked to check out new stuff you post…

Hey! Do you know if they make any plugins to assist with Search Engine Optimization? I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good gains.
If you know of any please share. Thank you!

Thanks for ones marvelous posting! I genuinely enjoyed reading it,
you’re a great author.I will always bookmark your blog
and will come back later in life. I want to encourage yourself to continue your great job,
have a nice day!

If some one wishes to be updated with newest
technologies therefore he must be go to see this website
and be up to date all the time.|

Can I simply just say what a comfort to uncover someone that genuinely knows what they are
talking about on the web. You certainly know how to bring a problem to light and
make it important. More and more people should look
at this and understand this side of the story. It’s surprising you are
not more popular since you surely have the gift.|

Woah! I’m really digging the template/theme of this site.

It’s simple, yet effective. A lot of times it’s very difficult to get that “perfect balance” between superb usability and visual appearance.

I must say that you’ve done a very good job with this.
In addition, the blog loads extremely fast for
me on Internet explorer. Superb Blog!|

Hey this is somewhat of off topic but I was wanting to know if blogs
use WYSIWYG editors or if you have to manually code with HTML.
I’m starting a blog soon but have no coding skills so
I wanted to get guidance from someone with experience.
Any help would be enormously appreciated!|

I could not resist commenting. Exceptionally well written!|

I am not sure where you are getting your info, but good topic.
I needs to spend some time learning much more or understanding more.
Thanks for great info I was looking for this info for
my mission.|

I pay a visit each day some sites and information sites to
read articles or reviews, but this website offers quality based writing.|

Hey there! I could have sworn I’ve been to this blog before
but after reading through some of the post I realized it’s
new to me. Nonetheless, I’m definitely delighted I found it and I’ll be book-marking and checking back often!

I really like your blog.. very nice colors & theme. Did you make this
website yourself or did you hire someone to do it for you?

Plz respond as I’m looking to create my own blog and would like to know where u
got this from. many thanks

It’s amazing in support of me to have a website, which is
good designed for my experience. thanks admin

I was wondering if you ever thought of changing the page layout of your site?
Its very well written; I love what youve got to say.
But maybe you could a little more in the way of content
so people could connect with it better. Youve got an awful
lot of text for only having one or two pictures.
Maybe you could space it out better?

Very good article. I will be experiencing a few of these
issues as well..

Nice respond in return of this query with firm arguments and describing all concerning that.

I’m gone to convey my little brother, that he should also visit this weblog on regular basis to get updated from most
up-to-date reports.

Leave a Reply