The LUV VM is now running BTRFS. I've setup a cron job to make read-only snapshots of the root filesystem. Such snapshots are atomic which means that the people who are making offsite backups of the server can rsync the MySQL data files without integrity problems. Also the filesystem has duplicate metadata to cover the case of corruption. The Dom0 is running Linux Software RAID-1 which saves us from a total disk failure but doesn't handle corruption at all. If data corruption hits the data blocks then we have to rely on offsite backups. BTRFS doesn't support duplicate data blocks (in any normal or desirable configuration) unlike ZFS which has the copies= configuration. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Tue, 12 Jan 2016 04:16:27 PM Russell Coker via luv-main wrote:
BTRFS doesn't support duplicate data blocks (in any normal or desirable configuration) unlike ZFS which has the copies= configuration.
David Sterba just released btrfs-progs 4.4 which has a new feature: # * mkfs.btrfs --data dup # # People asked about duplicating data on a single device for a long time. # There are no technical obstacles preventing that, so it got enabled with # a warning about potential dangers when the device will not do the # duplicated copies. # # See mkfs.btrfs section 'DUP PROFILES ON A SINGLE DEVICE'. The referenced manual page says: https://git.kernel.org/cgit/linux/kernel/git/kdave/btrfs-progs.git/tree/Docu... # DUP PROFILES ON A SINGLE DEVICE # ------------------------------- # # The mkfs utility will let the user create a filesystem with profiles that write # the logical blocks to 2 physical locations. Whether there are really 2 # physical copies highly depends on the underlying device type. # # For example, a SSD drive can remap the blocks internally to a single copy thus # deduplicating them. This negates the purpose of increased redunancy and just # wastes space. # # The duplicated data/metadata may still be useful to statistically improve the # chances on a device that might perform some internal optimizations. The actual # details are not usually disclosed by vendors. As another example, the widely # used USB flash or SD cards use a translation layer. The data lifetime may # be affected by frequent plugging. The memory cells could get damaged, hopefully # not destroying both copies of particular data. # # The traditional rotational hard drives usually fail at the sector level. # # In any case, a device that starts to misbehave and repairs from the DUP copy # should be replaced! *DUP is not backup*. -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
On Tue, 19 Jan 2016 09:33:03 PM Chris Samuel via luv-main wrote:
On Tue, 12 Jan 2016 04:16:27 PM Russell Coker via luv-main wrote:
BTRFS doesn't support duplicate data blocks (in any normal or desirable configuration) unlike ZFS which has the copies= configuration.
David Sterba just released btrfs-progs 4.4 which has a new feature:
# * mkfs.btrfs --data dup # # People asked about duplicating data on a single device for a long time. # There are no technical obstacles preventing that, so it got enabled with # a warning about potential dangers when the device will not do the # duplicated copies. # # See mkfs.btrfs section 'DUP PROFILES ON A SINGLE DEVICE'.
That's great news! Still no equivalent to copies=3 though...
# For example, a SSD drive can remap the blocks internally to a single copy thus # deduplicating them. This negates the purpose of increased redunancy and just # wastes space.
If you had a cryptsetup device on the SSD then it would be unable to remap the blocks as the block address goes into the encryption calculation.
# The traditional rotational hard drives usually fail at the sector level. # # In any case, a device that starts to misbehave and repairs from the DUP copy # should be replaced! *DUP is not backup*.
That depends on what you are doing. I have a collection of 3TB disks that I use for backup which each have between 20 and 100 bad sectors. I run them with BTRFS RAID-1 across 2 partitions and haven't had a problem with them. <100 bad sectors out of 3TB of disk space isn't much and as long as the number of bad sectors doesn't increase dramatically it works OK. The probability of one of those drives failing significantly is greater than the probability for a drive with no bad sectors. But I think it's still lower than the probability of me dropping one of those drives while carrying it around. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
participants (2)
-
Chris Samuel -
Russell Coker