I found I was subscribed to luv-main with a slightly incorrect address (different to what I was emailing as)... anyway...
parent transid verify failed on 699477344256 wanted 11799 found 12458
My btrfs lasted about a week before this happened.
I'm googling around to see what my options are... I have a btrfs send backup of my main subvolume but would lose a bunch of mythtv recordings that I'd rather keep.
Any suggestions appreciated!
All fixed. I restored from the btrfs send backup, and before I did that I copied my mythtv recordings off, and nothing appears to have been lost (although they could all be corrupt... copying them back now). The mysql database appears intact too, even though it would have been active right up until it crashed. The send/receive functions of btrfs are awesome. I have never seen an easier way of recovering before. The send/receive is well documented, but not the actual recovery when things go bad. I ended up re-creating a completely new btrfs volume and receiving the backup into that, which means the UUID's are different etc which is only a bit of a nuisance. I still don't know exactly what triggered the failure. My best guess is bcache, which was documented as not playing nicely with btrfs late last year (http://thread.gmane.org/gmane.comp.file-systems.btrfs/31018), although I'm not seeing any of those particular errors. I was using bcache in writeback mode, but switched back to writethrough for now. James
On Tue, 7 Oct 2014 02:53:50 AM James Harper wrote:
The send/receive functions of btrfs are awesome.
Watch out with 3.17 then, there are early reports that it's broken btrfs send. :-( -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
On Tue, 7 Oct 2014, Chris Samuel <chris@csamuel.org> wrote:
On Tue, 7 Oct 2014 02:53:50 AM James Harper wrote:
The send/receive functions of btrfs are awesome.
Watch out with 3.17 then, there are early reports that it's broken btrfs send.
:-(
My impression of btrfs send/recv is that the small number of bug reports is due to the small number of people who use that feature not the stability of it. Rsync works well on BTRFS so for my use I've found enough excitement in using a basic filesystem with snapshots and havn't felt the need to try send/recv. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Tue, 7 Oct 2014, Chris Samuel <chris@csamuel.org> wrote:
On Tue, 7 Oct 2014 02:53:50 AM James Harper wrote:
The send/receive functions of btrfs are awesome.
Watch out with 3.17 then, there are early reports that it's broken btrfs send.
:-(
My impression of btrfs send/recv is that the small number of bug reports is due to the small number of people who use that feature not the stability of it. Rsync works well on BTRFS so for my use I've found enough excitement in using a basic filesystem with snapshots and havn't felt the need to try send/recv.
So you would: . snapshot source . rsync to snapshot to backup medium . snapshot backup medium right? I think I could do that too, if send/receive does turn out to be broken. The send is just _so_ much faster it would be a shame not to stick with it. Although if I was backing up to an external location where network bandwidth was my biggest constraint then the performance of rsync would be less of a problem. James
On Tue, 7 Oct 2014, James Harper <james@ejbdigital.com.au> wrote:
So you would: . snapshot source . rsync to snapshot to backup medium . snapshot backup medium
right?
That's what I do.
I think I could do that too, if send/receive does turn out to be broken. The send is just so much faster it would be a shame not to stick with it.
Rsync is quite fast if you don't use the -c option and the average file size is reasonably large.
Although if I was backing up to an external location where network bandwidth was my biggest constraint then the performance of rsync would be less of a problem.
That's a large part of my backup. Another significant backup issue for me is some of my relatives who have 120G SSDs. Rsync from a SSD is mostly limited by the seek time of the backup disk, and when it's only 120G of data I don't need it to be really fast to complete before I leave. On Tue, 7 Oct 2014, "Peter Ross" <Petros.Listig@fdrive.com.au> wrote:
Means: Make rsync work like btrfs send/receive;-) and put filesystem specific code in it.
I am not sure whether this is a great idea.
I think that modern filesystems are getting complex enough that backup tools just can't operate properly without FS specific code.
Most of the time you will have the same filesystem on both ends. Then you can use zfs/btrfs etc. tools. Or rsync if it's not a COW system.
True, but that adds difficulty in migrating. For example my Internode quota is now 250G (was 150G last month), but I'm doing a backup of >400G of data over the Internet. AFAIK I couldn't change from using rsync backups to ZFS send/recv (even if I wanted to use ZFS at home) because I can't catch up with the backlog of data in any reasonable amount of time/money/effort. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
My impression of btrfs send/recv is that the small number of bug reports is due to the small number of people who use that feature not the stability of it. Rsync works well on BTRFS so for my use I've found enough excitement in using a basic filesystem with snapshots and havn't felt the need to try send/recv.
It seems rsync integration is on the list of projects https://btrfs.wiki.kernel.org/index.php/Project_ideas#Rsync_integration: " Rsync integration Not claimed - no patches yet - Not in kernel yet Now that we have code to efficiently find newly updated files, we need to tie it into tools such as rsync and dirvish. (For bonus points, we can even allow rsync to use btrfs's builtin checksums and, when a file has changed, tell rsync _which blocks_ inside that file have changed. Would need to work with the rsync developers on that one.) Update rsync to preserve NOCOW file status. Reference to other backup tools Not exactly an implementation of this, but I (A. v. Bidder) have patched dirvish to use btrfs snapshots instead of hardlinked directory trees. Discussion archived on the Dirvish mailing list. " James
Subject: Re: btrfs :(
On Tue, 7 Oct 2014 02:53:50 AM James Harper wrote:
The send/receive functions of btrfs are awesome.
Watch out with 3.17 then, there are early reports that it's broken btrfs send. :-(
Broken how? 3.16 had a bug in bcache where the writeback process would hang, so I did actually switch to 3.17-rc5 (debian experimental). Also 3.17 finally includes my patch to add support for my DVB card so I don't have to do the out-of-tree build for v4l. Both the send and receive were done on 3.17 A quick google says that 51f395ad4058883e4273b02fdebe98072dbdc0d2 might be the offending commit (https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg37877.html), although there isn't any followup to say if this is the case or not. If that is the case then 3.17-rc5 includes that commit so maybe I was just lucky? James
On Tue, 7 Oct 2014 04:04:48 AM James Harper wrote:
Watch out with 3.17 then, there are early reports that it's broken btrfs send.>
Broken how?
Mentioned on the linux-btrfs list: # After upgrading to kernel 3.17 btrfs send has stopped working. # # ERROR: send ioctl failed with -5: Input/output error # # The following message is printed by kernel: # # [75322.782197] BTRFS error (device sda2): did not find backref # in send_root. inode=461, offset=0, disk_byte=1094713344 # found extent=1094713344 # # btrfs inspect-internal inode-resolve -v 461 /u00/root.snapshot returns: # # /var/log/emerge-fetch.log # # After removing this file, the error moves on to another file. He tried some things from Chris Mason and then reported: # I just tried downgrading to 3.16.3 again. In 3.16.3 btrfs send # is working without any problem. Afterwards I upgraded again # to 3.17 and the problem reappeared. So the problem seems # to be kernel version related. Chris M. asked him to revert a specific commit also mentioned in this section of another issue that's emerged in 3.17, below. # Just tried it and I confirm filefrag's call to ioctl FS_IOC_FIEMAP # fails with -EEXIST. # # It's actually a known issue affecting any of the 3.17 RCs (except RC1). # The extent map manipulation/merging is broken for some cases. Try # with this 2 patches on top of 3.17-rcX: # # https://patchwork.kernel.org/patch/4929981/ # https://patchwork.kernel.org/patch/4945191/ # # Or, alternatively, reverting this patch: # https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=5... # # Someone else reported on this list a write/pwrite/writev failure with # errno EEXIST too (and apparently caused by the same reason). # # This broken extent map handling is serious IMHO, it can make fsync # log bogus extent items for example, amongst other possible bad and # weird things. The OP in this case reported that the patches fixed his issue, didn't comment on reverting this commit. Hopefully these can get tracked down quickly. ;-( cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
participants (3)
-
Chris Samuel -
James Harper -
Russell Coker