February 2012 - luv-main

RAID magic
by Russell Coker 11 Feb '12

11 Feb '12

Below is a magic entry I wrote to support Linux software RAID components. Does anyone here have a big-endian system that they can run software RAID on to test this? I suspect that the solution is to duplicate the section and change every "lelong" to a "belong". Also note that I want to recognise big endian devices on little endian systems and vice-versa. I'll file a Debian bug report about this as soon as I think I have big-endian sorted out. # Magic local data for file(1) command. # Insert here your local magic data. Format is described in magic(5). 4096 lelong 0xa92b4efc Linux Software RAID >4100 lelong x version %d >4112 belong x UUID=%8x: >4116 belong x \b%8x: >4120 belong x \b%8x: >4124 belong x \b%8x >4128 string x name=%s >4168 lelong x level=%d >4188 lelong x disks=%d -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

Re: state of ZFS and btrfs
by Craig Sanders 11 Feb '12

11 Feb '12

On Fri, Feb 10, 2012 at 06:53:23PM -0800, Daniel Pittman wrote: > > it's in ZFS. the catch is that it takes enormous amounts of > > memory....about 1GB RAM per TB of disk space IIRC to store the hashes > > for each block. I suspect the same catch would apply to other > > implementations. > > FreeBSD report that as ... optimistic, in the real world: quite probably. my figure of 1GB/TB was no better than a guess based on something half-remembered. > There are some resources that suggest that one needs 2GB per TB of > storage with deduplication [i] (in fact this is a misinterpretation of > the text). In practice with FreeBSD, based on empirical testing and > additional reading, it's closer to 5GB per TB. ouch. > I am inclined to take their comment seriously, given reports from the > few people I know who did try that. (OTOH, 64GB of RAM is US $379 for > four consumer grade sticks, $369 total, or $369 per 16GB stick? I thought 8GB & 16GB sticks were still stupidly expensive. > so you could realistically use that for a 12TB pool in hardware that > is reasonably affordable.) OTOH, there's probably lots better uses for that RAM on the same system. de-duping is, IMO, one of those things that sounds nice but that there's little practical use for. in most cases, it's cheaper and more effective to just add more disk than to add enough RAM to support it. craig -- craig sanders <cas(a)taz.net.au> BOFH excuse #40: not enough memory, go get system upgrade

1 0

Re: state of ZFS and btrfs
by Russell Coker 11 Feb '12

11 Feb '12

On Sat, 11 Feb 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > There are some resources that suggest that one needs 2GB per TB of > storage with deduplication [i] (in fact this is a misinterpretation of > the text). In practice with FreeBSD, based on empirical testing and > additional reading, it's closer to 5GB per TB. > > http://wiki.freebsd.org/ZFSTuningGuide#Deduplication > > I am inclined to take their comment seriously, given reports from the > few people I know who did try that. (OTOH, 64GB of RAM is US $379 for > four consumer grade sticks, so you could realistically use that for a > 12TB pool in hardware that is reasonably affordable.) You could use 64G of RAM for deduplication on 12TB of disk, that would save some amount of storage and also increase cache efficiency. But you could use the 64G of RAM for caching and just buy some extra disk. If you have a large number of DomUs then they probably aren't loaded up with a full KDE or GNOME environment. A basic DomU can have as little as 400M of data (when excluding /var) - based on inspecting one of my servers. So if I was to use deduplication for DomUs then I might need more than 24,000 to fill the 12TB of storage before deduplication. There are other ways that deduplication could really do some good, such as when a large MS-Word document is attached to an email that is sent to everyone in a company. Of course you would probably have to remove the Delivered-To: header to get a good result with that (I'm guessing that ZFS deduplication only works with block aligned data). But there are other solutions to that problem like using my maildir-bulletin program or some other delivery agent that is specific to the case of mass local mail-outs. Given modern storage prices I find it difficult to imagine a situation where deduplication does some real good but where it can't be done better at the application level. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

Re: RAID-1 synchronisation
by Russell Coker 11 Feb '12

11 Feb '12

On Sun, 5 Feb 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > Yes, it does good. For something like RAID-5 that has enough > information to detect which device is wrong you can find and correct > problems with the system. With a RAID-1 you can know that a device is > returning bad data. You mean RAID-6. RAID-5 is no better than RAID-1 when it comes to determining which one of the disks is returning corrupt data. On Sun, 5 Feb 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > Yes. This is the substantial advantage that BTRFS and ZFS have over > device-level RAID and LVM. The stronger checksums are useful because > they can be checked inline with lower I/O cost, and the intimate > knowledge of allocated space means you can check more quickly. Both > very attractive features. https://btrfs.wiki.kernel.org/ The above URL lists mirroring as "Additional features in development, I don't think I can use this on serious servers for a while. https://btrfs.wiki.kernel.org/articles/u/s/i/Using_Btrfs_with_Multiple_Devi… mkfs.btrfs in Debian/Unstable apparently supports RAID (the above URL has background information), so I will test it out. On Sun, 5 Feb 2012, James Harper <james.harper(a)bendigoit.com.au> wrote: > One thing it does do for you is 'touch' unused blocks, and finding that > those are bad now rather than later is better IMO. Also, verifying > consistency and finding that you have a silent corruption problem early > can only be a good thing. This is especially important for RAID5 without > battery backed write cache as it can detect the RAID5 write-hole > (http://en.wikipedia.org/wiki/RAID_5_write_hole) Maybe write-intent > bitmaps get around this these days though? The only difference between the RAID-5 write hole and the issue of mismatched RAID-1 devices is what happens on a device failer. It's a lot worse for RAID-5 so that's a good reason for doing such checks on a RAID-5 array, or just using RAID-6. RAID-6 allows detecting and correcting the situation where one disk has corrupt data and all disks are working and it allows detecting the situation where the disks aren't consistent if there has been a single disk failure. Just don't use RAID-5. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

8 15

Making IPV6 routes Persistent in RedHat
by Leigh Sharpe 10 Feb '12

10 Feb '12

Hi All, I'm having trouble getting RHEL to re-instate persistent IPV6 routes on reboot. I have the routes set up in /etc/sysconfig/network-scripts/route6-eth0 but they aren't getting applied on startup. But when I do an ifdown followed by an ifup, or a 'service network restart' I get the following message: RTNETLINK answers: No route to host ..Which is fair enough, as it seems that the IPV6 network isn't actually ready when the route is added. Has anybody got this working before? Is there somewhere else I should be adding the routes? Cheers, Leigh.

4 5

LUV hardware library
by Russell Coker 07 Feb '12

07 Feb '12

I'm starting the LUV hardware library tonight. I will bring a briefcase full of computer stuff which has no monetary value to me (it's stuff that either I got for free or which I can't use or expect to sell) but which would be really useful to anyone who has systems which need such parts. It's probably too late to expect anyone to bring any hardware to donate tonight, but one thing I could use that many people have is a bunch of anti- static bags. Please bring a pocket full if you have some lying around. For donations, any hardware which could be useful for someone and which fits in your pocket will do. Some items of particular interest are: Small USB storage devices. Probably most people are like me and have no use for anything less than 1GB. But some people can really use them (like the relative who took all my 128MB USB devices for MP3s to play in their car). PCI cards. As PCIe is in all new systems there isn't a lot of use for PCI apart from all the old systems that are in production. PCI Ethernet cards are really good for making old systems into routers. IDE disks and small SATA disks. Probably most people use nothing less than 300G nowadays, but there are lots of uses for 40G disks (such as making old systems into small servers with RAID-1). Please don't bring me more than 2 disks at one time without prior arrangement, due to the mass it would be inconvenient to unexpectedly receive many disks. USB keyboards and mice. USB cables and extension cables. RAM, anything faster than PC-133 (I think I've got a lifetime supply of that) and bigger than 128MB. ECC RAM is useful, but please make sure it's labelled as such - otherwise ECC RAM can look like broken RAM... Also if you attend tonight and find that some type of small and old hardware that you need isn't in the collection then feel free to make requests. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

mysql connection problems
by Russell Coker 06 Feb '12

06 Feb '12

4 4

Re: RAID-1 synchronisation
by Craig Sanders 05 Feb '12

05 Feb '12

On Sat, Feb 04, 2012 at 10:44:51AM -0800, Daniel Pittman wrote: > I suppose it is something more akin to the fsck that ext* still have, > but XFS omits: you can totally trust the journal to recover for you, are you sure you don't mean btrfs there instead of xfs? xfs_check and xfs_repair definitely exist. btw, when btrfs is stable/production-system-ready then the authors are correct - fsck won't be necessary. same as it isn't necessary on ZFS. they both do (or will do in the case of btrfs) continuous consistency checking and file-level checksumming. and the btrfs authors are right to focus on improving btrfs itself rather than getting too side-tracked by an fsck tool. craig -- craig sanders <cas(a)taz.net.au> BOFH excuse #374: It's the InterNIC's fault.

1 0