Backups


The easiest way to backup things is to schedule everything to run at midnight when no one is on the system. Stop thinking like that! You should be able to backup at any time of the day at any load without penalty to the end user. You never know when you need a backup or when the trigger could be accidentally pulled. Here are a couple ways to handle backups AND do it in a sane way.

 

Backing up Zeo

Backing up zeo is so easy, there is no excuse not to do it. Using traditional backup software/strategies is unnecessary and very likely to be corrupted in any moderately busy setup. No worries, though. Repozo has you covered. Put that baby on a cron job and take a vacation. How often to backup? As much as possible so that it doesn't affect the system. The longer you wait the harder the backup is on the system. Don't be afraid to set it up for every hour, although 3 times a day may be sufficient for most moderately sized systems. If you have multiple zeos make sure you are backing up all of them! A sample bash script for that is here.

 

If you are using buildout, collective.recipe.backup creates a nice wrapper for repozo. Creating backups is than as easy as 'bin/backup' and 'bin/snapshotbackup'.

 

DO make sure these backups are on a different disk, not only to make the backup go faster but also because if your disk fails you are SOL if they are on the same disk.

 

If these processes are killing your IO and/or CPI, look at controlling them with nice and ionice.

 

Backing up Other Resources with rsync

For other resources, such as files stored with FileSystemStorage and currently blob support (which is being fixed), we need another source of backup. There are lots of things you can pay for, especially if you are on a windows box but if you are stubborn like the rest of us there is rsync. Luckily there is a great tutorial on setting these up.

 

If you are having performance spikes when running rsync, all hope is not lost. First try prefixing all of your commands with nice and ionice (-c3). Next, trying doing your backups in chunks. Then try getting some leverage out of the --bwlimit parameter (start at 6000). Remember that any disk activity will negatively impact your zeo performance not only from the disk i/o but also because the operating system will be trying to cache that backup disk activity in RAM, effectively stealing it from your zopes. If nice doesn't help you get past this, do the backup in pieces. Instead of taking the backup from a single root folder, break down the subfolders and do some pieces at a time.