LXC-Backup
Attention: This blog post is from 2014, but still seems to get a lot attention. Many things have changed. LXD has been released and there is ZFS support in LXC, which is far superior to this method. I’ll post an update soon.
If you are using Linux containers for deployment, you might want to backup a whole container instead of just the application data. This comes with a down side. If your application has some sort of state and is saving data in the file system or a database, you’ll have to shutdown the container to make a backup. Otherwise you might lose data, since some of the data might still be in RAM and RAM won’t be saved to the backup. At least not with this method. So this method should only be used if the container has not to be up 24/7. But usually the backup is pretty fast and only takes a few seconds.
Backups with duply
I’m using duply for the backups. duply is a frontend for duplicity. The advantage of duply lies within the configuration management. Configs (and some more stuff) for all the duply backups are stored in /etc/duply/. duply also allows easy execution of pre and post scripts, which allows us to stop and start containers for the backup. Incremental backups are pretty much standard now and speed things up a little bit. I’ll use the invoices container as an example.
This is how /etc/duply/lxc_invoices/ looks like:
├── conf
├── exclude
├── post
└── pre
The conf file
The conf might look like this:
GPG_KEY='disabled'
GPG_PW='_GPG_PASSWORD_'
TARGET='file:///backup/lxc_invoices'
SOURCE='/var/lib/lxc/invoices/'
MAX_FULL_BACKUPS=2
Basically this tells duply to performe a backup of /var/lib/lxc/invoices (this is where all the files of the container are saved) to /backup/lxc_invoices. Encryption of the backup is deactivated and there should be not more than two full backups.
If we’d execute duply /etc/duply/lxc_invoices backup right now, duply would backup all files of the container, but we couldn’t be sure if the backup would actually be consistent. Therefore we’ll have to stop the container right before the backup and start it right after the backup. This is where the files post and pre come into play.
Pre and post backup scripts
pre is the shell script that is executed before the backup is performed:
#!/bin/bash
set -eu
LXCSHUTDOWN='/usr/bin/lxc-stop'
LXCWAIT='/usr/bin/lxc-wait'
LXCATTACH='/usr/bin/lxc-attach'
CONTAINER=invoices
$LXCATTACH -n $CONTAINER -- /usr/bin/apt-get clean
$LXCSHUTDOWN -n $CONTAINER
$LXCWAIT -n $CONTAINER -s 'STOPPED'
post is the shell script that is executed after the backup:
#!/bin/bash
set -eu
LXCSTART='/usr/bin/lxc-start'
LXCWAIT='/usr/bin/lxc-wait'
CONTAINER=invoices
$LXCSTART -n $CONTAINER -d
$LXCWAIT -n $CONTAINER -s 'RUNNING'
Now duply /etc/duply/lxc_invoices backup stops the container, performs a backup and starts the container after the backup. Everything should be up an running again.
May the cron be with you
Since people are working with the invoice system, backups shouldn’t be perfomed during working hours - even though backups only take around 30 seconds and full backups of the invoice container usually won’t take longer than a minute. But this depends on the amount of data you have to save. The invoice container is not very big with circa 1GB.
To perform the backup at quater past three in the night you’ll have to add the following line to your /etc/crontab:
15 3 * * * root /usr/bin/duply /etc/duply/lxc_invoices backup
At dajool I give duply a five minute window for each container backup, which is more than enough for all the containers - even the big ones.
Sources at GitHub
You can find all the example files on GitHub as public domain. If you have questions or suggestions, just open a ticket on GitHub. Have fun!
UPDATE:
The GitHub repository of lxc-duply-backup no longer only hosts the example files and has advanced into a toolkit to create packages (Deb and RPM) for the scripts and configs needed to backup your LXC. Take a look at the repository README for more information.