- luv-main - luv.asn.au

Re: state of ZFS and btrfs
by Russell Coker 11 Feb '12

11 Feb '12

On Sat, 11 Feb 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > There are some resources that suggest that one needs 2GB per TB of > storage with deduplication [i] (in fact this is a misinterpretation of > the text). In practice with FreeBSD, based on empirical testing and > additional reading, it's closer to 5GB per TB. > > http://wiki.freebsd.org/ZFSTuningGuide#Deduplication > > I am inclined to take their comment seriously, given reports from the > few people I know who did try that. (OTOH, 64GB of RAM is US $379 for > four consumer grade sticks, so you could realistically use that for a > 12TB pool in hardware that is reasonably affordable.) You could use 64G of RAM for deduplication on 12TB of disk, that would save some amount of storage and also increase cache efficiency. But you could use the 64G of RAM for caching and just buy some extra disk. If you have a large number of DomUs then they probably aren't loaded up with a full KDE or GNOME environment. A basic DomU can have as little as 400M of data (when excluding /var) - based on inspecting one of my servers. So if I was to use deduplication for DomUs then I might need more than 24,000 to fill the 12TB of storage before deduplication. There are other ways that deduplication could really do some good, such as when a large MS-Word document is attached to an email that is sent to everyone in a company. Of course you would probably have to remove the Delivered-To: header to get a good result with that (I'm guessing that ZFS deduplication only works with block aligned data). But there are other solutions to that problem like using my maildir-bulletin program or some other delivery agent that is specific to the case of mass local mail-outs. Given modern storage prices I find it difficult to imagine a situation where deduplication does some real good but where it can't be done better at the application level. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

Re: RAID-1 synchronisation
by Russell Coker 11 Feb '12

11 Feb '12

On Sun, 5 Feb 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > Yes, it does good. For something like RAID-5 that has enough > information to detect which device is wrong you can find and correct > problems with the system. With a RAID-1 you can know that a device is > returning bad data. You mean RAID-6. RAID-5 is no better than RAID-1 when it comes to determining which one of the disks is returning corrupt data. On Sun, 5 Feb 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > Yes. This is the substantial advantage that BTRFS and ZFS have over > device-level RAID and LVM. The stronger checksums are useful because > they can be checked inline with lower I/O cost, and the intimate > knowledge of allocated space means you can check more quickly. Both > very attractive features. https://btrfs.wiki.kernel.org/ The above URL lists mirroring as "Additional features in development, I don't think I can use this on serious servers for a while. https://btrfs.wiki.kernel.org/articles/u/s/i/Using_Btrfs_with_Multiple_Devi… mkfs.btrfs in Debian/Unstable apparently supports RAID (the above URL has background information), so I will test it out. On Sun, 5 Feb 2012, James Harper <james.harper(a)bendigoit.com.au> wrote: > One thing it does do for you is 'touch' unused blocks, and finding that > those are bad now rather than later is better IMO. Also, verifying > consistency and finding that you have a silent corruption problem early > can only be a good thing. This is especially important for RAID5 without > battery backed write cache as it can detect the RAID5 write-hole > (http://en.wikipedia.org/wiki/RAID_5_write_hole) Maybe write-intent > bitmaps get around this these days though? The only difference between the RAID-5 write hole and the issue of mismatched RAID-1 devices is what happens on a device failer. It's a lot worse for RAID-5 so that's a good reason for doing such checks on a RAID-5 array, or just using RAID-6. RAID-6 allows detecting and correcting the situation where one disk has corrupt data and all disks are working and it allows detecting the situation where the disks aren't consistent if there has been a single disk failure. Just don't use RAID-5. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

8 15

Making IPV6 routes Persistent in RedHat
by Leigh Sharpe 10 Feb '12

10 Feb '12

Hi All, I'm having trouble getting RHEL to re-instate persistent IPV6 routes on reboot. I have the routes set up in /etc/sysconfig/network-scripts/route6-eth0 but they aren't getting applied on startup. But when I do an ifdown followed by an ifup, or a 'service network restart' I get the following message: RTNETLINK answers: No route to host ..Which is fair enough, as it seems that the IPV6 network isn't actually ready when the route is added. Has anybody got this working before? Is there somewhere else I should be adding the routes? Cheers, Leigh.

4 5

LUV hardware library
by Russell Coker 07 Feb '12

07 Feb '12

I'm starting the LUV hardware library tonight. I will bring a briefcase full of computer stuff which has no monetary value to me (it's stuff that either I got for free or which I can't use or expect to sell) but which would be really useful to anyone who has systems which need such parts. It's probably too late to expect anyone to bring any hardware to donate tonight, but one thing I could use that many people have is a bunch of anti- static bags. Please bring a pocket full if you have some lying around. For donations, any hardware which could be useful for someone and which fits in your pocket will do. Some items of particular interest are: Small USB storage devices. Probably most people are like me and have no use for anything less than 1GB. But some people can really use them (like the relative who took all my 128MB USB devices for MP3s to play in their car). PCI cards. As PCIe is in all new systems there isn't a lot of use for PCI apart from all the old systems that are in production. PCI Ethernet cards are really good for making old systems into routers. IDE disks and small SATA disks. Probably most people use nothing less than 300G nowadays, but there are lots of uses for 40G disks (such as making old systems into small servers with RAID-1). Please don't bring me more than 2 disks at one time without prior arrangement, due to the mass it would be inconvenient to unexpectedly receive many disks. USB keyboards and mice. USB cables and extension cables. RAM, anything faster than PC-133 (I think I've got a lifetime supply of that) and bigger than 128MB. ECC RAM is useful, but please make sure it's labelled as such - otherwise ECC RAM can look like broken RAM... Also if you attend tonight and find that some type of small and old hardware that you need isn't in the collection then feel free to make requests. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

mysql connection problems
by Russell Coker 06 Feb '12

06 Feb '12

4 4

Re: RAID-1 synchronisation
by Craig Sanders 05 Feb '12

05 Feb '12

On Sat, Feb 04, 2012 at 10:44:51AM -0800, Daniel Pittman wrote: > I suppose it is something more akin to the fsck that ext* still have, > but XFS omits: you can totally trust the journal to recover for you, are you sure you don't mean btrfs there instead of xfs? xfs_check and xfs_repair definitely exist. btw, when btrfs is stable/production-system-ready then the authors are correct - fsck won't be necessary. same as it isn't necessary on ZFS. they both do (or will do in the case of btrfs) continuous consistency checking and file-level checksumming. and the btrfs authors are right to focus on improving btrfs itself rather than getting too side-tracked by an fsck tool. craig -- craig sanders <cas(a)taz.net.au> BOFH excuse #374: It's the InterNIC's fault.

1 0

one free desktop pc for people who need it
by Rock 26 Jan '12

26 Jan '12

hi all I have one free desktop pc for people who need it. it is in good condition and works smoothly. I tested Windows XP and several Linux distributions on it and all of them working perfectly. it can be used for checking email, document processing and small games in daily work. now I would like to give it to someone need one desktop for daily using or Linux study. the details are: CPU:Intel Pentium4 2.80GHz Motherboard:Sapphire RS300-SB200 RAM:768 MB Harddisk:WD 80GB+40GB Graphics:Nvidia GeForce FX 5500 ( 256 MB ) ATAPI DVD R/W Sound:ATI IXP150 AC'97 Audio Controller Network:realtek RTL8139. it does not contain the monitor, keyboard and mouse. if interested you can send email to me. yours Rock

1 0

reverse proxy with authentication
by James Harper 25 Jan '12

25 Jan '12

I need to secure a bunch of web apps on separate internal servers for accessing from the internet, and it seems that a reverse proxy is probably a decent way of doing this. The problem is going to be authentication as I want to put front end security on the proxy (so unauthenticated users can't touch my apps with their own dubious security) but some of the apps require their own username and password, potentially different to the proxy. This won't work using conventional web security obviously, so I'll need to get a little creative. What I was thinking is something that goes like this: . User goes to apps.company.com.au/auth and enters username + password . Proxy validates credentials and sets a cookie . The Proxy validates the cookie for each request then 'eats' it before passing the request to the internal server . Any access without the set cookie gets redirected to the /auth page for authentication . Apps are accessed via apps.company.com.au/appname My google-fu is weak just lately though and I can't find anything that might do what I want... the reverse proxy part is fine (squid or apache) but not the front end authentication layer. The docs for mod_proxy (which I've used before) don't seem to suggest that I can do it that way. Any suggestions? Thanks James

2 2

reading raw disk of Dell perc RAID adapter
by James Harper 19 Jan '12

19 Jan '12

I have been handed a raw disk of a Dell perc (I think) RAID adapter. It was a RAID1 array but the machine is broken and can't be repaired in the short term. I was expecting to be able to track down the partition table at some offset into the disk and then identify the partitions etc but of course it's not that easy. I've found a few instances of the 0x55 0xaa partition signature at the end of a sector but only one actually appears to be a partition table (about 64MB from the start of the disk), but even then it doesn't look right. I then tracked down an NTFS signature (the disk is out of a Windows machine but I'm trying to get the data off on a Linux machine) but ntfs-3g and other tools says it's corrupt. So I'm guessing that one of the following is happening: 1. The RAID controller crapped all over the disk before the system failed and it is actually corrupt 2. The RAID is actually RAID0 (I was assured it was RAID1 but you know how these things go) 3. Dell have some non-linear mapping of partitions dmraid doesn't know anything about it, which kind of makes sense as the Dell raid controllers are normally not fakeraid. Any suggestions? Thanks James

2 1

Linux and music
by Kathy Reid 13 Jan '12

13 Jan '12

Hi everyone, Happy new year! If anyone has any advice for me on the following, it would be very much appreciated. Problem statement: I'm reliant on Windows for music. I own an iPod shuffle, and am reliant on iTunes to download music to this device. What I would like to do is buy another, non-Apple music player, and fulfill the following requirements; 1. Run a music program within Linux (Ubuntu / Unity) 2. Download music legally within Linux 3. Load my music player from within the music program 4. Convert the music from iTunes/Apple format and have it available to the Linux music program. What are my options? Regards, Kathy

7 9