http://en.wikipedia.org/wiki/Backup_rotation_scheme Does anyone know of a good script for managing btrfs snapshots for different frequencies of backups? Something like grandfather-father-son or tower of hanoi would do. Here's the long form: To cover the fumble-fingers case of backups (as opposed to hardware failure/theft) I want to have automatically managed btrfs snapshots. I was thinking of something like every 15 minutes for the last 24 hours, every 8 hours for the previous 30 days, and every day for the 100 days before that. Note that this schedule won't necessarily be the best for me (every 15 minutes for the last hour would do for me) but I'm dealing with users who won't necessarily call me quickly when they do the wrong thing and it's probably best to use the same configuration for everyone. Anyway this will end up being something like 96+90+100==286 snapshots at any time. I don't expect btrfs to have any problem handling this as the systems in question will mostly be used for Chromium, LibreOffice, and email. I expect that there will be some performance overhead and a definite space overhead, but nothing that a 120G SSD with a single active user can't cope with. The problem for me is managing it. I'm sure that I could write my own scripts to do this, but I'm concerned about bugs. I'd rather use someone else's script that has been tested and works at least some of the time. ;) -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell Coker <russell@coker.com.au> writes:
Does anyone know of a good script for managing btrfs snapshots for different frequencies of backups? [...] like every 15 minutes for the last 24 hours, every 8 hours for the previous 30 days, and every day for the 100 days before that.
Actually taking the backups is trivial - a fifteen minute cron job. At the end of that job, you expire any snapshots that are superfluous to your rotation policy -- e.g. if it's four hourlies and seven dailies, then you expire the fifth hourly &c My immediate inclination would be to rip the code for that out of rsnapshot. However IIRC that relies heavily on the snapshots being named <rotation>.<Nth>, which is why at Cyber we have an in-house Python script for expiring superfluous ZFS snapshots. If you're interested, let me me know and I'll see if I can publish it (it's not my code so I have to ask first.)
On Sun, 9 Dec 2012, "Trent W. Buck" <trentbuck@gmail.com> wrote:
Russell Coker <russell@coker.com.au> writes:
Does anyone know of a good script for managing btrfs snapshots for different frequencies of backups? [...] like every 15 minutes for the last 24 hours, every 8 hours for the previous 30 days, and every day for the 100 days before that.
Actually taking the backups is trivial - a fifteen minute cron job.
Yes.
At the end of that job, you expire any snapshots that are superfluous to your rotation policy -- e.g. if it's four hourlies and seven dailies, then you expire the fifth hourly &c
That's the hard part.
My immediate inclination would be to rip the code for that out of rsnapshot. However IIRC that relies heavily on the snapshots being named <rotation>.<Nth>, which is why at Cyber we have an in-house Python script for expiring superfluous ZFS snapshots.
I want to have human-readable names so that I can instruct people over the phone. I want to say "go to ~/2012-12-09-06:15 and you'll find the old version of your file". Even working out which rotation.nth directory corresponds to the last time the file was good would be too much effort to do over the phone. Also for the cases where there's more than one user I'll probably make snapshots in the home directory for each user. Currently I don't run a workstation that has enough users for this to be any sort of problem, the maximum number of workstation users that are important enough for a backup is 2.
If you're interested, let me me know and I'll see if I can publish it (it's not my code so I have to ask first.)
Please do that. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell Coker <russell@coker.com.au> writes:
If you're interested, let me me know and I'll see if I can publish it (it's not my code so I have to ask first.)
Please do that.
OK, here we go. Hopefully it's intelligible. There's no support or anything, but feel free to send feedback or patches my way, either on- or off-list as you deem appropriate.
Hey, On 09/12/2012, at 5:58 PM, Russell Coker <russell@coker.com.au> wrote:
The problem for me is managing it. I'm sure that I could write my own scripts to do this, but I'm concerned about bugs. I'd rather use someone else's script that has been tested and works at least some of the time. ;)
I believe this is what Snapper[1] was designed and developed to do for openSUSE. I have no idea if it's been ported to any other distro yet, though. Cheers, Avi [1] http://en.opensuse.org/Portal:Snapper
On Sun, 9 Dec 2012, Avi Miller <avi.miller@gmail.com> wrote:
I believe this is what Snapper[1] was designed and developed to do for openSUSE. I have no idea if it's been ported to any other distro yet, though.
Thanks for the suggestion, but it seems a fairly heavy-weight solution. It's got a snapperd that runs from dbus (a daemon?) and a command line utility to control it. It supports LVM and Ext4 as well as BTRFS which is a lot of overhead when you just want BTRFS. I'd like to have a simple command-line utility that can run from cron with parameters to specify the number of snapshots to keep. Support for both BTRFS and ZFS would be nice, but something that supports only BTRFS would be fine. Support for LVM and Ext4 is a lot of extra complexity that I will never use. I will probably be more likely to write my own tool in shell script or Perl than to use Snapper. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On 9 December 2012 21:30, Russell Coker <russell@coker.com.au> wrote:
Thanks for the suggestion, but it seems a fairly heavy-weight solution. It's got a snapperd that runs from dbus (a daemon?) and a command line utility to control it. It supports LVM and Ext4 as well as BTRFS which is a lot of overhead when you just want BTRFS.
Maybe not much use for you, however here is a solution that is LVM specific (and also uses a naming convention that is compatible with the Samba Volume Shadow Copy interface) https://wiki.samba.org/index.php/Rotating_LVM_snapshots_for_shadow_copy -- Brian May <brian@microcomaustralia.com.au>
participants (4)
-
Avi Miller -
Brian May -
Russell Coker -
trentbuck＠gmail.com