-----Original Message----- From: luv-main-bounces@luv.asn.au [mailto:luv-main-bounces@luv.asn.au] On Behalf Of Russell Coker Sent: Thursday, 23 May 2013 5:24 PM To: luv-main@luv.asn.au Subject: Re: backups & power [was Re: uncorrectable read errors]
On Thu, 23 May 2013, Robin Humble <rjh+luv@cita.utoronto.ca> wrote:
nothing clever, just manually. overly manually. every week or so I power on the backup box and run some rsyncs. it'd be possible to automate it (script that runs on power-up) but I haven't bothered.
I actually kinda like a >>1day gap between backups as it gives me a window to retrieve files that I deleted by mistake. a fraction of a day is a bit short to realise that a file is gone and to look for it in the backups.
http://etbe.coker.com.au/2012/12/17/using-btrfs/
I'm currently using BTRFS snapshots for that sort of thing. On some of my systems I have 100 snapshots stored from 15 minute intervals and another 50 or so stored from daily intervals. The 15 minute intervals capture the most likely possibilities for creating and accidentally deleting a file. The daily once cover more long-term mistakes.
That's pretty neat. I do the same with Windows, but it's nice to see that Linux supports this now too. Windows would not support a 15 minutes snapshot interval though - docs say no more than 1 an hour or something like that. Recovering data under windows is as simple as right click then show previous versions and you select which snapshot you want to look at. Samba can do this too. How does performance fare with lots of snapshots? Windows goes with the concept that the snapshot holds the changed data, so first-write becomes a read-from-original + write-original-data-to-snapshot-area + write-new-data-to-original[1]. This reduces first-write performance but subsequent writes suffer no penalty, and means no fragmentation and throwing a snapshot away is instant. I think LVM actually writes the changed data into the snapshot area (still may require a read from original if the write isn't exactly the size of an extent) but I can't remember for sure. If so it means the first -write is faster but subsequent writes are still redirected to another part of the disk, and your data very quickly gets massively fragmented and recovery in the event of a booboo is a bitch if lvm metadata goes bad (from experience... I just gave up pretty much immediately and restored from backup when this happened to me[2]!). How does btrfs do it internally?
incremental backups would work better, but so far I've been too lazy to work out a way to filter out fewGB tv shows that I've watched and deleted and don't want in any incrementals.
On a BTRFS or ZFS system you would use a different subvolume/filesystem for the TV shows which doesn't get the snapshot backups.
I'm getting more and more excited about btrfs. I was looking around at zfs but it didn't end up meeting my needs. I'm still testing ceph and xfs is currently recommended for the backend store, btrfs is faster but has known issues with ceph, or at least did last time I read the docs and so is not currently recommended. James [1] with the default snapshot provider. Windows can outsource this function to a SAN or whatever else (could even be a Xen backend running zfs/btrfs!!) so obviously ymmv. [2] when the docs for clustered lvm say you can't use snapshots[3] then you can't use snapshots. Don't think you can. You will be wrong :) [3] this was years ago. It has changed since then, with some restrictions