Skip to main content

Ralsina.Me — Roberto Alsina's website

Personal Backups with rdiff-backup

What is rdiff-backup

Quot­ing their web page:

rdif­f-back­up backs up one di­rec­to­ry to an­oth­er, pos­si­bly over a net­work. The tar­get di­rec­to­ry ends up a copy of the source di­rec­to­ry, but ex­tra re­verse diffs are stored in a spe­cial sub­di­rec­to­ry of that tar­get di­rec­to­ry, so you can still re­cov­er files lost some time ago. The idea is to com­bine the best fea­tures of a mir­ror and an in­cre­men­tal back­up. rdif­f-back­up al­so pre­serves sub­di­rec­to­ries, hard links, dev files, per­mis­sion­s, uid/gid own­er­ship, and mod­i­fi­ca­tion times. Al­so, rdif­f-back­up can op­er­ate in a band­width ef­fi­cient man­ner over a pipe, like rsync. Thus you can use rdif­f-back­up and ssh to se­cure­ly back a hard drive up to a re­mote lo­ca­tion, and on­ly the dif­fer­ences will be trans­mit­ted. Fi­nal­ly, rdif­f-back­up is easy to use and set­tings have sen­si­cal de­fault­s.

What We Are Going To Do

In this ar­ti­cle I will not give a tu­to­ri­al on rdif­f-back­up, I will just use it as a sim­ple per­son­al back­up tool. For this, I could use a mir­ror­ing tool, which would be OK, but rdif­f-back­up is both easy to get and easy to use, and most im­por­tan­t, I know how it works ;-) 1

So, here's the gen­er­al plan:

  • Back­­up your home fold­er, or some part of it.

  • Do it ef­­fi­­cien­t­­ly, both in CPU us­age and in disk us­age.

  • Keep a his­­to­ry of changed files (and on­­ly of changed files). In fac­t, rdif­f-back­­up will on­­ly keep the changed pieces (the "delta") along with the full ver­­sion.

  • Do it au­­to­­mat­i­­cal­­ly

  • Do it on­­to a fold­er owned by an­oth­er user, so a mo­­ment of in­­san­i­­ty (rm -Rf / as a reg­u­lar user) or a tro­­jan does­n't de­stroy the back­­up­s.

What This Is Not

This is not a re­al, or se­ri­ous back­up so­lu­tion!

  • We have no dis­­as­ter re­­cov­­ery strat­e­­gy

  • If the sys­tem re­al­­ly goes to heav­en, the back­­ups do too

  • If root goes in­­sane, he can wipe all back­­ups

So, why both­er? Be­cause this is easy, and not hav­ing any back­ups is worse. And tell me the truth... do you have any back­ups of your per­son­al fold­er­s?

I am of the opin­ion that if the re­al so­lu­tion is hard, it's of­ten a good idea to spend the ef­fort it takes to re­al­ly solve the prob­lem.

But I am al­so of the opin­ion that some­times, if the re­al so­lu­tion is too hard, most peo­ple is not go­ing to care enough about the prob­lem to ac­tu­al­ly solve it. And in those cas­es, there is room for an al­most good enough non-­so­lu­tion.

The Real Stuff

If you want to per­form re­al, se­ri­ous back­up­s, I rec­om­mend you look at Aman­da or Bac­u­la or some oth­er of the pletho­ra of back­up so­lu­tions avail­able, de­fine a pol­i­cy, and start do­ing off­site (or at least of­f-sys­tem) back­up­s.

Some are good, some are very good, al­most all of them are bet­ter than this ar­ti­cle's sug­ges­tion (in some way at least ;-)

Getting rdiff-backup

In their home­page you can find Fe­do­ra RPM­s, and bi­na­ries for some oth­er OSs or dis­tri­bu­tion­s.

I am us­ing Red Hat 9, and com­piled it from the .s­r­c.rpm with­out prob­lem­s, af­ter get­ting li­brsync from the same page, al­so in .s­r­c.rpm for­mat.

Backing Up Your Stuff With rdiff-backup

Step 1

As root, cre­ate a fold­er some­where, where your back­ups will go. I will use /home­/back­ups in this ex­am­ple, you can use what­ev­er you wan­t, but:

  • If it's on an­oth­er disk, it will be faster

  • It will use a large amount of space

The space thing is like this: The back­up will be just as large as what­ev­er you want to back­up. Then it will get larg­er with time, un­til it be­comes, may­be, many times as large, de­pend­ing on how many old­er ver­sions you want to keep of your da­ta.

If you are not root, then you can still do it, but the back­up will lose some of its prop­er­ties (for ex­am­ple, a tro­jan or a mis­take would be able to delete it!)

Step 2

Decide what to backup. For example, I want to backup /home/ralsina/projects, where all my in-progress stuff is stored, and /home­/ralsi­na/.kde, where the configuration (and more) of KDE is saved.

Step 3

Let's try it. Here's a shell script for my ex­am­ple:

#!/bin/sh

# Where the backups go
target=/home/backups
mkdir -p "$target"

# Just put all the folders you want backed here. If they
# have strange characters in the name, just put them in
# single quotes. If you are not sure, quote them anyway.

for folder in /home/ralsina/projects /home/ralsina/.kde
do

        # If the folder being backed up contains ".."
        # something bad is going on

        if echo "$folder" | grep ..
        then
                continue
        fi

        mkdir -p $target/$folder
        rdiff-backup $folder $target/$folder
done
chown root.root -R $target
chmod o-w -R $target

The pur­pose of the chown and chmod at the end is to make sure no one can mod­i­fy the back­up­s. Think of it as stor­ing them in a vault.

If you are back­ing up stuff from var­i­ous user­s, that will be wrong since it's mak­ing root own ev­ery­thing.

Here's an al­ter­na­tive:

#!/bin/sh

# Where the backups go
target=/home/backups
mkdir -p "$target"

# Just put all the folders you want backed here. If they
# have strange characters in the name, just put them in
# single quotes. If you are not sure, quote them anyway.


for folder in /home/ralsina/projects /home/ralsina/.kde
do
        # If the folder being backed up contains ".."
        # something bad is going on

        if echo "$folder" | grep ..
        then
                continue
        fi

        mkdir -p $target/$folder
        rdiff-backup $folder $target/$folder
done
chown root.root $target
chmod 700 $target

This ver­sion pre­serves the own­ers of ev­ery­thing, but it "lock­s" the en­trance to the back­up fold­er. This way, the back­ups are safe from ac­ci­dents, but on­ly root can re­store stuff... which may be too in­con­ve­nien­t.

You can just re­move the chown/ch­mod com­mand­s, and that ver­sion will keep back­up­s, but not pro­tect them against the us­er him­self.

So, take your back­up scrip­t, save it some­where with a rea­son­able name like, in my case, /us­r/lo­cal/bin/ralsi­na-back­up, make it ex­e­cutable 2

Restoring

To get back the last ver­sion of a file from the back­up, you could just copy it from the back­up di­rec­to­ry.

You can re­store both re­cent and old ver­sions us­ing the --re­store-as-of op­tion of rdif­f-back­up.

For ex­am­ple:

rdif­f-back­up --re­store-as-of (time) /home­/back­up­s/home­/ralsi­na/pro­ject­s/­doc.txt

Will re­store the /home­/ralsi­na/pro­ject­s/­doc.txt file from mo­ment (time), where (time) has this for­mat (quot­ing the rif­f-back­up doc­s):

  1. the string "now" (refers to the cur­rent time)

  2. a se­quences of dig­it­s, like "123456890" (indi­­cat­ing the time in sec­onds af­ter the epoch)

  3. A string like "2002-01-25T07:00:00+02:00" in date­­time for­­mat

  4. An in­­ter­­val, which is a num­ber fol­lowed by one of the char­ac­ters s, m, h, D, W, M, or Y (indi­­cat­ing sec­ond­s, min­utes, hours, days, week­s, mon­th­s, or years re­spec­­tive­­ly), or a se­ries of such pairs. In this case the string refers to the time that pre­ced­ed the cur­rent time by the length of the in­­ter­­val. For in­­s­tance, "1h78m" in­­di­­cates the time that was one hour and 78 min­utes ago. The cal­en­­dar here is un­­so­phis­ti­­cat­ed: a month is al­ways 30 days, a year is al­ways 365 days, and a day is al­ways 86400 sec­ond­s.

  5. A date for­­mat of the form YYYY/M­M/D­D, YYYY-M­M-D­D, MM/D­D/YYYY, or MM/D­D/YYYY, which in­­di­­cates mid­night on the day in ques­­tion, rel­a­­tive to the cur­rent time­­zone set­t­ings. For in­­s­tance, "2002/3/5", "03-05-2002", and "2002-3-05" all mean March 5th, 2002.

  6. A back­­up ses­­sion spec­i­­fi­­ca­­tion which is a non-neg­a­­tive in­­te­ger fol­lowed by 'B'. For in­­s­tance, '0B' spec­i­­fies the time of the cur­rent mir­ror, and '3B' spec­i­­fies the time of the 3rd new­est in­­cre­­men­t.

Limiting the Age of Backups

The scripts giv­en above would keep old ver­sions of files for­ev­er. That is not prac­ti­cal, since it would re­quire a huge amount of disk space.

To lim­it that, you can use the --re­move-old­er-than op­tion.

For ex­am­ple, if we used

rdiff-backup --remove-older-than 2W $folder $target/$folder

It would re­move files old­er than two week­s. The for­mat of the date for re­moval is the same as the one giv­en above for restor­ing.

Automating The Process

You can take ad­van­tage of cron to make this all run au­to­mat­i­cal­ly. Just cre­ate an en­try in root's crontab to do this ev­ery night, or what­ev­er.

Here's an ex­am­ple dai­ly back­up at mid­night:

0 0 * * * /usr/local/bin/ralsina-backup 2>&1 | mail ralsina

This sends a mail to ac­count ralsi­na in the lo­cal box con­tain­ing the out­put of the com­mand.

If you are not fa­mil­iar with cron, you can do it by hand. I sug­gest you get fa­mil­iar with it, or use a graph­i­cal cron man­age­ment tool.

Offering Automatic Backups For All Users

A more ad­vanced back­up script could read a cer­tain file (say "back­upthis") from each user's fold­er, and get from there the in­for­ma­tion on what to back­up. Here's a quick and dirty ver­sion:

#!/bin/sh
for homefolder in /home/*
do
        if [ -f $homefolder/backupthis ]
        then
                while read $homefolder/backupthis
                do

                        # If the folder being backed up contains ".."
                        # something bad is going on

                        if echo "$folder" | grep ..
                        then
                                continue
                        fi

                        mkdir -p $target/$homefolder/$folder
                        rdiff-backup $homefolder/$folder $target/$homefolder/$folder
                done < $homefolder/backupthis
        fi
done

Notice that for this script, if I wanted to backup /home/ralsina/projects, in my backupthis file I would have to put only projects.

This is in­ten­tion­al, and the goal is that users can on­ly back­up stuff from their own home fold­er­s.

This script is not meant for ar­bi­trary in­put. It's not re­al­ly well writ­ten. If a us­er re­al­ly re­al­ly want­ed, he could find ways to make it do weird and dan­ger­ous stuff ( It's quick and dirty as I said ;-)

Other Possible Uses

You could use al­most this ex­act pro­ce­dure for keep­ing two home fold­ers sync'd over the In­ter­net, thus let­ting you use both com­put­ers and have all your da­ta at hand. To do that, read rdif­f-back­up's in­for­ma­tion about us­ing it over ssh. How­ev­er, Uni­son is prob­a­bly a bet­ter idea.

Final Words

I Hope you find this use­ful. rdif­f-back­up is a very cool tool, and has helped me well in a cou­ple of sit­u­a­tion­s. It's very ef­fi­cien­t, and sim­ple to use.

How­ev­er, if you start us­ing these script­s, I strong­ly sug­gest you learn more about rdif­f-back­up from its doc­s, and its wi­ki


1

It's al­so writ­ten on python, which means I can un­der­stand how it does what it does, and that's a bonus.

2

chmod u+x /us­r/lo­cal/bin/ralsi­na-back­up

employment background check / 2011-12-27 23:23:


Hi very nice article


Contents © 2000-2020 Roberto Alsina