dcsimg

Comprehensive Desktop Backup

A few hard drives and rsync make a simple and effective onsite backup system. Scott Granneman shows you how to keep your desktop data safe.

Backup is vitally important, and you should do it as regularly as possible. There’s nothing worse than losing valuable data. However, individual backup needs vary, so different strategies and software packages are required.

This column describes the onsite backup strategy I use. This technique is supplemented (not replaced) with my offsite backup strategy, which uses a combination of rsync and Amazon’s S3, which was described here a short time ago.

I have four IDE hard drives, each 320 GB in size, and each dedicated to a particular kind of data (yes, a move to 500 GB SATA drives is imminent). The name of each drive corresponds to its mount point in /media, as shown in the table.

TABLE ONE: A map of the hard drives on the author’s desktop system
Drive name Mount point
data /media/data
movies /media/movies
music /media/music
music-rock /media/music-rock

In addition to the four data drives, I have four backup drives, each with its own corresponding mount point, too: /media/data-copy, /media/movies-copy, /media/music-copy, and /media/music-rock-copy.

Each drive is in its own enclosure. After trying many different enclosures, I finally found what I consider to be the best of breed: the AMS VENUS DS-2316CBK Aluminum 3.5″ USB & 1394 Black External Enclosure. Priced at about $50 at NewEgg, this is a sturdy enclosure that supports both USB 2 and FireWire. Best of all, there’s a fan on the bottom to keep things cool.

But I was faced with one big problem: as I steadily increased the number of drives I owned, I had to increase the number of enclosures, which meant additional power, as well as an enormous rat’s nest of power and USB cables behind my desk. There had to be a better way. Finally, I purchased another two enclosures, also created by AMS. At about $115 at NewEgg, the AMS VENUS T4U DS-2340UBK 3.5″ USB 2.0 External Enclosure holds four internal hard drives, with one power cord and one USB 2 connection for all the drives.

In one T4U, I put data, movies, music, and music-rock; in the other, data-copy, movies-copy, music-copy, and music-rock-copy. When I buy a new hard drive, I use fdisk to create a single partition on the drive. After that, I format the new partition as ext3 with the command mkfs.ext3 /dev/sda1. By default, ext3 reserves 5 percent of the hard drive for emergency usage. On a 320 GB hard drive, that’s a ridiculous amount. 2 percent is more reasonable, which the following command puts in place: tune2fs-m 2 /dev/sda1.

For ease of reference and mounting, I give the hard drive a label via tune2fs-L music /dev/sda1. Even though I assign a label, I prefer to use the drive’s UUID (it’s unique ID) in /etc/fstab. Discovering the drive’s UUID isn’t that hard.

$ sudo vol_id /dev/sda1ID_FS_UUID=081914a5-7365-4ec3-b284-fe8e0f23e258

In fstab, I add an entry for the drive (the entry is wrapped below to fit the column width):

UUID=081914a5-7365-4ec3-b284-fe8e0f23e258   /media/music ext3 rw,user,noauto 0 0

I must also create a mount point with the command sudo mkdir /media/music. I can now mount the new drive with mount /media/music. Of course, if I reboot (a very rare occurrence), KDE auto-mounts all the drives, making my life far easier.

The drives whose names end in “-copy” are backups of the original drives. To handle the backups, I use rsync. My script for the music drives, which copies /media/music to /media/music-copy, looks like this:

rsync --verbose --progress --stats \  --recursive --times --perms --links \  --delete /media/music/ /media/music-copymv /media/music-copy/111aaa_Music \  /media/music-copy/111aaa_Music_Copy

The latter line runs mv because a long time ago I was careless and stupidly deleted some files off of a “-copy” drive instead of its master drive. To prevent that, I created a file on data called 111aaa_Data and one on data-copy called 111aaa_Data_Copy, with similarly-named files on the other drives. When rsync runs, it copies 111aaa_Music from the master to the copy, so I use mv to rename the file.

That’s my onsite backup strategy. Future refinements include larger SATA drives, as well as the introduction of a T4U to hold hard drives dedicated to backups of my movies and connected to a Linux media center running Myth. Now, back your stuff up!

Comments on "Comprehensive Desktop Backup"

registerhater

rsnapshot – same building blocks, and you get historical info too, not just one copy

bedge

I second rsnapshot. It provides rotating incremental backups that minimize space by using links and provides easy retreival and archive browsing.

jwaldram

Don’t forget the –link-dest= option! You get what appear to be full backups at the cost of the directory entries and only the changed files. You still have previous backups!

See:
http://www.mikerubel.org/computers/rsync_snapshots/
http://rsync.samba.org

johnfisher

This is quite similar to the strategy I use at work to backup a small software office. Once we dumped the big tape drive, I took some old PIII generic boxes and created two backup servers, one for local and one at the home office in another town. (We have no IT dept, thank goodness, so individuals have to be somewhat responsible.)

There is a canonical user and filename regex for all machines. Each development server tarballs the important areas and ships them over to the local backup server.
Each user is allowed a home on the backup server. (desktop data is not mission critical) users are responsible for tarballing their own data and shipping it to their home on the backup server. I use a Ubuntu/Debian package called “simple network backup” to do incremental backups of my own desktop, but a simple cron job would also do.
The backup server untars the mission critical tarballs and rsyncs them to the backup server offsite over an encrypted link. We do daily, weekly, monthly, yearly archiving using cron jobs.

Important KISS principles:
Use standard Linux components- perl, ftp, ssh, tar, rsync, cron
Use (otherwise) obsolete machines with cheap non-RAID hard drives
Avoid automatic restore – just save directory trees and expect to manually restore if necessary
Critical data is stored in three places, including off-site
No tape
No IT dept
No backup of OS – its easier to reload than restore
If a machine is awake and has a file of the right filename at the right location, back it up, else ignore
Avoid RAID

JF

robinbowes

Why avoid RAID?

johnfisher

Robin, I had a bad experience at a previous job with an wildly-overcomplicated RAID system with failover, but I also just plain think its too complex for simple stuff like backups. It has a place of course for high-speed servers and such, but my thinking about backups is keep it simple and dumb. I prefer hot spares to RAID enabled boxes for smaller servers too. When in doubt pick the simpler low-performance low-maintenance solution.
JF

jester

I can’t agree with you more about keeping things simple stupid; however, RAID has its place in KISS backups. If you’re only talking about mission critical files (meaning relatively little storage space required), then RAID-1 is your best friend. Though it costs more (a $15 PCI card for older machines, and another hard drive), you get a lot of redundancy, in fact 100% times the number of drives in the RAID. And if you RAID the entirety of both (or all) drives including your OS, then if a drive dies you can just take that bad one out and keep on chugging while you order another drive. Then there is no down time while you recover.

Also, what are peoples’ thoughts on Unison as a backup program?

johneeboy3

BackupPC is a fantastic backup tool, which gets precious little air time.

http://backuppc.sourceforge.net/

Key points:
* It’s GPL
* It’s rsync-based, and written in perl.
* It will back up Windows, Linux or OSX machines
* It ‘pools’ identical files, saving massive amounts of space on the backup host
* It compresses all backed up files.
* It’s extremely flexible. Basically anything can be (re)configured.
* It facilitates backing up fixed (desktop) and often disconnected (laptop) machines.
* It has a great web-based interface, allowing you to drill down to individual files in the backup as well as see the complete change history of a backed up file. On top of that, restores only take a few clicks and the file(s) are restored automatically back onto the target.

I’ve been using backuppc in a production environment for 3+ years and it ‘just works’ and never fails.

I currently have over 920GB of files backed up across more than 10 machines. Due to pooling and compression, the backup pool is only about 160GB!!!

Simply fantastic and highly recommended. It’s saved my IT life on numerous occasions. I’ve reviewed most backup systems and nothing else comes close to BackupPC.

Cheers

John

johneeboy3

Alex,

Unison was not really designed as a backup tool. It is primarily a tool for maintaining mirrored data.

In that (latter) capacity, it’s great.

I used it for a few months as a backup tool, but it has problems. It was a while ago now, so I’m sorry but I can’t be more specific as to why – my bad memory!

See my post above about BackupPC. It’s fantastic.

Cheers

John

malahal

>> No backup of OS – its easier to reload than restore

I use gentoo stage4 backup and it was lot easier to just boot with liveCD and run tar extract than installing gentoo (maybe installing other distro’s too). More over, installing all the packages you need and customizing them or configuring them for you would take lot more time.

drokmed

>> No backup of OS – its easier to reload than restore

> I use gentoo stage4 backup and it was lot easier to just boot with liveCD and run tar extract than installing gentoo

Agreed. When I’m done building a server, and it’s ready for use, I always boot it with SystemRescueCD live cd, and backup the partition table and partitions (Partimage) to a writable DVD. Takes about an hour.

I can build a new replacement server in about an hour, online and ready for data restore.

pwiesen

I’ve had great success getting back up and running after a hard drive failure with a DVD created using Mondo Restore. Mondo along with rsnapshot has saved me many headaches.

mikemtnbikes

>By Alex Rawls November 29th, 2007 at 5:10 pm
>
>I can’t agree with you more about keeping things >simple stupid… Then there is no down time while >you recover.
>
>Also, what are peoples’ thoughts on Unison as a >backup program?
>
Unison rocks. I’ve been using it for years to keep my home dir on my laptop and office computers in sync. Rock solid. Quick. Logical. I recommend it.

Greetings! I know this is kind of off topic but I was wondering if you knew where I could locate a captcha plugin for my comment form? I’m using the same blog platform as yours and I’m having problems finding one? Thanks a lot!

Hf9fgj jcnmngldgxrd, [url=http://lafycjhodenc.com/]lafycjhodenc[/url], [link=http://iqemmpqwetrf.com/]iqemmpqwetrf[/link], http://dorrrifywrxw.com/

Leave a Reply