Improved backups with rsync and ZFS

Today backups saved someones bacon again. A customer messaged me and asked if I could restore a file from yesterday. Luckily this is a piece of cake (just like it should be).

Our production systems are not yet using ZFS and this will definetly change in the future. But our backup systems are using ZFS extensively. No matter how backups are performed on-site, the backups are transfered to the backup systems via rsync. After that a ZFS snapshot is created. This way we don't have to pay attention to how the backup was performed on-site. We just mount the snapshot from that day and restore the files needed. Only transfering ZFS snapshots directly would be easier and I definitely see that in our future.

So, how is it working?

We've got a script in /etc/cron.daily that does something like this:

#! /bin/sh

# backup voyager
/usr/bin/rsync -avz -e ssh --delete voyager:/backup/. /tank/backup/voyager/.
/sbin/zfs snapshot tank@$(date +"%Y-%m-%d_%H-%M-%S")

A zfs list -t snapshot on cochrane - one of the backup systems - would show something like this:

brejoc@cochrane ~> sudo zfs list -t snapshot  
NAME                       USED  AVAIL  REFER  MOUNTPOINT  
tank@now                      0      -   483G  -  
…
tank@2016-02-19_07-50-59  79,4M      -   703G  -  
tank@2016-02-20_08-14-51  49,2M      -   704G  -  
tank@2016-02-21_07-56-23    16K      -   705G  -  

Restoring something from one of those snapshots couldn't get easier. The most current backup can always be found in /tank/backup/voyager/. It's just the currently mounted pool.
If one of the older backups is needed, you can simply mount the ZFS snapshot like this: sudo mount -t zfs tank@2016-02-19_07-50-59 /mnt/snapshot/ - and then restore the file(s) from /mnt/snapshot. That's it!