Home » Debian, Featured, Headline, Linux, Recovery, Shell

Subversion backup

26 March 2009 5 Comments
Subversion Backup

Subversion Backup

Maintaining a subversion repository can be a hard task if you don’t work with the right tools. Today, when technology advance daily, we are still vulnerable to “fate”: crashed harddrives, corrupt RAM, network outages, power failure or other “evil” problems. Even if you are a conscientious administrator you will not be missed by “Murphy Laws”, only if you are a “bastard” lucky guy (I’m a bastard lucky administrator from hell but you are not). In the next article I will try to show you, how-to create a good backup of your subversion repositories.

We can create two types of backups: incremental and full.

You have 2 possibilities to create incremental backups: subversion hooks and scheduling them with crontab. If you are paranoid you can use subversion hooks and you can create a incremental backup at every commit (Do you really want that ?). Depending by your system and your repository size, doing backups at every commit can cost you enormously, and if your hardware resources are not so “advanced” I don’t recommend you to do that.

Doing incremental backup at every commit for paranoid people.

Subversion have something beautiful functionality named hooks. This hooks are triggered by repository events such as:

pre and post new revision
pre,post, start commit
pre, post lock, unlock repository

and probably other.

To fetch a subversion incremental backup after a commit we should add a post-commit script file in all hook folders from your repositories. For example if you have your repositories in /home/svn/ then your hook folders should be in /home/svn/[repositories_names]/hooks/ . post-commit file will be called by subversion server every time when someone is finishing to commit on server. Also post-commit script file is called with 2 parameters: repository name and repository revision.

post-commit script

1
2
3
4
5
6
7
8
9
10
11
12
13
#!/bin/bash
 
BACKUP_DIR="/var/backups/svn/"
REPOSITORY_DIR="/home/svn/repository"
REPO=$1
REV=$2
 
if [ ! -d $BACKUP_DIR"/"$REPO ]; then
        mkdir -p $BACKUP_DIR"/"$REPO
    fi
 
 
svn-backup-dumps -r $REV $REPOSITORY_DIR"/"$REPO $BACKUP_DIR"/"$REPO

Copy post-commit script in /home/svn/*/hooks/ and create /var/backups/svn/

Now you are ready … just commit a file to repository.

Doing incremental backups every night.

This configuration is simple. We just need to add a script in /etc/cron.daily/ and backup will be taken every morning at 6 (under my Debian system).
The script will search in repository folder to find every repository what we have configured. After it will backup this repositories.

inc-backup.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#!/bin/bash
 
BACKUP_DIR="/backup/repository-inc/"
REPOSITORY_DIR="/home/svn/repository"
 
#search in repository folder to find all the repository names
ls -Al --time-style=long-iso $REPOSITORY_DIR | grep '^d' | awk '{print $8}' | while read line
do
#create the backup repository folder if we don't have it.
    if [ ! -d $BACKUP_DIR"/"$line ]; then
        mkdir -p $BACKUP_DIR"/"$line
    fi
 
#takeing backup (10 revisions in 1 dump)
    svn-backup-dumps --deltas -z -c 10 $REPOSITORY_DIR"/"$line $BACKUP_DIR"/"$line
done

The script will calculate delta between backups and repository and it will backup only the difference. Also will compress the resulting file (-z) what will contain 10 subversion revisions (-c 10). I think is important to have dumps very granulated and not 1000 revisions in 1 file, but this is your decision.

To recover from a incremental backup just run

svnadmin load /path/to/reponame < /var/backup/svn/repo/repo1.dump

Doing full backups

The story is the same as the above section, just the script is different and cron folder is /etc/cron.weekly/. Also for full backup we will use the svn-hot-backup script provided by svn. Hot backups are not easy to do because doing a straight copy of the repository can generate a faulty backup unless you cut all users access (what is not such a good thing to do). Resulting backup is a full working copy of the Subversion repository what can be deployed (just copy) over a crashed one.

full-backup.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#!/bin/bash
 
BACKUP_DIR="/backup/repository"
REPOSITORY_DIR="/home/svn/repository"
#search in repository folder to find all the repository names
ls -Al --time-style=long-iso /home/svn/repository/ | grep '^d' | awk '{print $8}' | while read line
do
    if [ ! -d $BACKUP_DIR"/"$line ]; then
        mkdir $BACKUP_DIR"/"$line
    fi
 
    #Getting revision number
    REVISION=`cat $REPOSITORY_DIR"/"$line"/db/current" | awk '{print $1}'`
 
    #Archive last backup
    tar -czf $BACKUP_DIR"/"$line"-last.tar.gz" $BACKUP_DIR"/"$line"/"
 
    #Dangerous :)
    if [ -n "$BACKUP_DIR" ]; then
	rm -rf $BACKUP_DIR"/"$line"/*"
    fi
 
    #Check to see if exists a hot backup
    if [ -d $BACKUP_DIR"/"$line"/"$line"-"$REVISION ]; then
        echo "Skipping Backup ! Backup Already Exists"
    else
        echo "Doing backup for "$line
        /usr/bin/svn-hot-backup $REPOSITORY_DIR"/"$line $BACKUP_DIR"/"$line
    fi
 
done

The script will also archive the latest full backup before taking a new one.

My recommendation is to archive all the files over a network on a NFS server and don’t backup them on the same server.Also don’t forget to get a full backup even if you already have incremental backups, because the best solution is a diversified one.

Good luck.


5 Comments »

  • xxxx said:

    Just a little remark:
    ls -lA

    My ubuntu machine(more or less default setup)
    ls -lA gives different output if the script is invoked from crontab or shell.

    The print $8 refers to wrong column in the ls command.
    Therefore put in the parameter –time-style=XXXX for ls command.

  • admin (author) said:

    Thanks for your remark. Is true. I think for different type of distributions default “ls” parameters are not the same. Debian is using
    “–time-style=long-iso” by default.

    Regards

  • millibyte said:

    Thanks for providing this info – just what I needed!

  • millibyte said:

    And here’s my little contribution in the form of two fixes for the full-backup.sh script:

    1) Change the ln -Al line to use $REPOSITORY_DIR
    2) The line
    if [ $BACKUP_DIR !="" ]; then
    causes an error (at least on my Ubuntu system). A better line would be:
    if [ -n "$BACKUP_DIR" ]; then

    Cheers

  • admin (author) said:

    Yes, true! I discovered this problem later and I changed the code with
    [code]
    if [ "$BACKUP_DIR" !="" ]; then
    [/code]

    But your solution is better.

    Thanks

    Regards

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.