| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

Backups

Page history last edited by Elizabeth Leddy 13 years, 8 months ago

The easiest way to backup things is to schedule everything to run at midnight when no one is on the system. Stop thinking like that! You should be able to backup at any time of the day at any load without penalty to the end user. You never know when you need a backup or when the trigger could be accidentally pulled. Here are a couple ways to handle backups AND do it in a sane way.

 

Backing up Zeo

Backing up zeo is so easy, there is no excuse not to do it. Using traditional backup software/strategies is unnecessary and very likely to be corrupted in any moderately busy setup. No worries, though. Repozo has you covered. Put that baby on a cron job and take a vacation. How often to backup? As much as possible so that it doesn't affect the system. The longer you wait the harder the backup is on the system. Don't be afraid to set it up for every hour, although 3 times a day may be sufficient for most moderately sized systems. If you have multiple zeos make sure you are backing up all of them! A sample bash script for that is here.

 

If you are using buildout, collective.recipe.backup creates a nice wrapper for repozo. Creating backups is than as easy as 'bin/backup' and 'bin/snapshotbackup'.

 

DO make sure these backups are on a different disk, not only to make the backup go faster but also because if your disk fails you are SOL if they are on the same disk.

 

If these processes are killing your IO and/or CPI, look at controlling them with nice and ionice.

 

Backing up Other Resources with rsync

For other resources, such as files stored with FileSystemStorage and currently blob support (which is being fixed), we need another source of backup. There are lots of things you can pay for, especially if you are on a windows box but if you are stubborn like the rest of us there is rsync. Luckily there is a great tutorial on setting these up.

 

If you are having performance spikes when running rsync, all hope is not lost. First try prefixing all of your commands with nice and ionice (-c3). Next, trying doing your backups in chunks. Then try getting some leverage out of the --bwlimit parameter (start at 6000). Remember that any disk activity will negatively impact your zeo performance not only from the disk i/o but also because the operating system will be trying to cache that backup disk activity in RAM, effectively stealing it from your zopes. If nice doesn't help you get past this, do the backup in pieces. Instead of taking the backup from a single root folder, break down the subfolders and do some pieces at a time.

Comments (2)

Nejc Zupan said

at 3:54 am on Dec 20, 2009

Buildout recipe collective.recipe.backup (http://pypi.python.org/pypi/collective.recipe.backup/1.3) creates a nice wrapper for repozo. Creating backups is than as easy as 'bin/backup' and 'bin/snapshotbackup'.

Elizabeth Leddy said

at 2:25 pm on Apr 13, 2010

thanks! added to the main text

You don't have permission to comment on this page.