- luv-main - luv.asn.au

btrfs/ZFS, sans raid and bitrot
by Noah O'Donoghue 01 Jul '14

01 Jul '14

Hi All, After reading about bitrot and feeling guilty for storing my most valuable data on cheap drives (although with backups!) I've been thinking about moving to something more resilient. My current setup is a Ubuntu laptop, with 2 external drives. 1X 2TB ext4 for data storage 1X 3TB ext4 for backup (using Crashplan commercial backup software). My question, is if I change the first drive to btrfs or ZFS, will I gain resiliency from bitrot? My understanding is I need 2 drives in at least a RAID 1 to get automatic healing from bitrot, but if I at least use a filesystem with check summing support then I will be able to at least restore my affected files from my Crashplan backups (which are compressed then checksummed and regularly checked for errors automatically) and I won't have the risk of my main drive corrupting my backups, because the read will FAIL if it doesn't pass the checksum. Is my understanding correct? Cheers, Noah

5 12

Re: btrfs/ZFS, sans raid and bitrot
by Russell Coker 01 Jul '14

01 Jul '14

On Tue, 1 Jul 2014 10:03:44 Rohan McLeod wrote: > Noah O'Donoghue wrote: > > Hi All, > > > > After reading about bitrot and feeling guilty for storing my most > > valuable data on cheap drives (although with backups!) I've been > > thinking about moving to something more resilient. > > Out of curiosity I googled "bitrot" and whilst there seems to be some > usage of "bitrot" in relation to RAM; > mostly it seems to be in the context of storage media. > As a novice I found: > > http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows > -inside-next-gen-filesystems/ http://en.wikipedia.org/wiki/Write_Anywhere_File_Layout That article claims that ZFS is the oldest of the "next generation filesystems". WAFL did it first and NetApp (the developer of WAFL) sued Sun alleging patent violation in ZFS. > informative; but apart from a suggestion that it might be related to > 'cosmic rays' and thermal magnetic effects; > couldn't seem to find > (a) a definition which is a measure of bitrot and > (b) actual measures of this phenomenon in various media and differing > conditions. > > Presumably as a probabilistic phenomenon; bitrot might be defined in > terms of the half-life of the data ? http://research.cs.wisc.edu/adsl/Publications/corruption-fast08.html The above paper is the best reference I've seen. Half-life isn't a good measure as you can expect to lose ~50 sectors at a time on a TB+ disk. On Tue, 1 Jul 2014 12:29:39 Peter Ross wrote: > For Russell: Have you seen this? > > https://wiki.freebsd.org/ZFS > > The first TODO entry is about file(1) and magic. That is about ZFS dump files (the output of "zfs send") not the block devices. As I have never run zfs send and don't have any immediate plans to do so this hasn't been a concern for me. Thanks for the suggestion though, I've attached it to the Debian bug report. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

2 2

hardware library
by Russell Coker 30 Jun '14

30 Jun '14

Here is a list of some things that I think would be good for the hardware library, if you have something that matches which you don't need then please bring it to a LUV meeting to donate it. Also just bring random stuff that's small and light. DDR2 and DDR3 RAM - I think there is plenty of older RAM in the library already. Laptop RAM. Not many people want it but those who do want it badly. It also works in lots of modern printers. USB Flash storage. While most of us get enough freebies from trade shows I'm sure that some people need more. It's small and light so I don't mind carrying them around. 64bit PCs. I'm not planning to carry spare machines to regular meetings, but I'm happy to store a few in case someone needs one and bring them to a meeting on demand. Post to the list first if you plan to bring a PC to the meeting. ATI PCIe video cards. NVidia has driver issues (either a lack of features or non-free depending on which driver you choose). Small Ethernet switches that are quiet. Even a 10baseT switch is very handy if it's small and light. USB mice. Micro-USB cables suitable for connecting Android devices (and many other things) to PCs. USB power supply as used for phones and tablets. Android devices. Even devices that run Android 2.1 can be useful if they aren't broken. More recent Android devices can be useful even if they are a bit broken (EG a cracked screen still leaves an Android device as a portable computer with a camera). Accessories for phones (please clearly label which phone they belong to). There are multiple standards for things like external microphones for phones, some old phones are unreasonably expensive to buy parts for. Batteries are useful if they can take at least 50% original charge, please wrap batteries in plastic to avoid the risk of short-circuiting. Here are some things that I personally need: A US style extension cable that goes from a PC PSU to a monitor. It's not a very important thing for me but would be nice to have. As few PCs have the socket for such a cable nowadays there should be a few unused out there. A quad-core PC with 4*DDR2 sockets that can take full height PCIe cards. SATA disks with many bad sectors. The ideal would be to have about 0.1% of reads fail. I'm doing some research into error recovery. Please clearly write "BAD" on such disks to avoid unfortunate mistakes. Note that when I am finished experimenting with such disks I will use a hammer to make them unreadable - just in case the disk problems made it impossible to delete some of your data. USB keyboards. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

2 1

zfs magic
by Russell Coker 28 Jun '14

28 Jun '14

I'm working on writing /etc/magic entries for ZFS. Below is the start that I've made. It uses XDR for the header which has pascal strings with a 4 byte big-endian length. I thought that should be "pstring/L" but the following pstring types are the ones that worked. Also running "file -s -k /dev/sd?" doesn't work, is the -k option expected to work on a device node? 0x4010 belong 0x20 >0x4014 pstring/l version ZFS store version >>0x4020 belong 8 >>>0x4024 belong x %d >>>0x403a pstring/h name >>>>0x4048 pstring/l x pool name %s -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

Webcam Brand/Model Suggestions
by David 27 Jun '14

27 Jun '14

TLDR: I am running out of time to buy a webcam for use with Skype on Debian, can anyone suggest a brand or model that is known to work? I'm asking for tips because I am on a tight timeline and the task is 5 hours drive away at a relative's house, where I'm going to install Debian 7. They have requested Skype, I've not used it nor even webcams before. I went to the local computer shop and plugged in a webcam and saw 0c45:62c0 in dmesg. But checking at home the Debian IRC bot "judd" says "Unknown device from unknown vendor". So I'm looking for tips to at least purchase something that will work. (I'm at work now so I cannot test anything.) Thanks for any assistance David

4 5

Interface bonding and ADSL
by Toby Corkindale 24 Jun '14

24 Jun '14

Hey all, Does anyone have real-world experience of using linux's interface bonding on public networks? (In the bandwidth-aggregation mode, not the redundancy mode) I was wondering how I could make the following setup work: * Rent a VPS in Melbourne with four IP addresses * Get four (or just two) ADSL connections wired up to home * Have your VPS connect four VPN connections from itself back to each of your home IPs. * Bond all four interfaces together * Create a fifth VPN connection, this time going over the bonded-virtual-interface between VPS and home, and then configure your home server to use that link as the default route? It sounds pretty messy and I'm not sure it'd actually work in practice; the routing tables would be hell to get right. Are there any guides already out there? Cheers, Toby

8 14

autofs and iptables
by Bill 22 Jun '14

22 Jun '14

Hello All, With autofs on Red Hat/Centos, the" /net -hosts" special map allows all NFS resources exported by all accessible NFS servers to get mounted under the /net directory without explicitly mounting each one of them. E.g. accessing /net/host1 will instruct autofs to mount all available resources on host1. At first, it didn't work. E.g. if I run cd /net/host1 I have always got "No such file or directory". After running tcpdump to analyse the packets, I discovered that this particular feature uses sunrpc (port 111) and a random UDP port (30000 above). The problem is how I set iptables rule for this random port. I know I can specify a port range like --dport 30000:60000, but this is not a safe way to configure a firewall. Cheers, Bill

5 4

free adsl 2+ modem
by cory seligman 17 Jun '14

17 Jun '14

Does anyone want a Tenda D820B ADSL 2+ modem? I have NBN now, so it's surplus. It's a modem only, not a router, so it needs to be managed by a router running PPPoE to work. I had it running with a WRT54G running OpenWRT for years - worked fine. Pickup in Brunswick West

1 1

Re: Telstra ADSL and Linux, a couple of dumb questions.....
by zlinw＠mcmedia.com.au 17 Jun '14

17 Jun '14

Erik Christiansen <dvalin(a)internode.on.net> said, >Here in Tecoma (Dandenong Ranges), my ISP (Internode) provides ADSL2+. >Is it a rural exchange which would limit it to ADSL1, or the 5 km line >length? >Erik Im practice it would be line length. As far as I am aware ADSL uses line frequecies up to around 1mhz, ADSL 2 to around 2.2 mhz. Special 1.27mm copper carrier cable has a loss at 1.6mhz of around 30db for 5 kilometres. One would expect most customer cable would be around double that. Such a loss would put receive levels for ADSL2 at 5000 metres well down into the noise (-50 to -60dbm). Even for ADSL you would need real good copper cable for it to work at 5000 metres. Lindsay

5 9

Telstra ADSL and Linux, a couple of dumb questions......
by zlinw＠mcmedia.com.au 16 Jun '14

16 Jun '14

I recently shifted, although still very much in rural Victoria I am now in ADSL range of an exchange (just over 5000 metres cable distance). I am considering getting the Telstra S+ bundle, this being a phone and ADSL broadband with a 50g alocation a month on a 24 month contract. I am fairly happy with this bundle it will save me around $4 to 500 dollars a year and giving __much__ higher data allowance. Is there any issues with this, will it work OK into a Linux system. For instance I found with Virgin moble broadband this system had to be enabled on a Windows system, something I do not have with network access. If it helps I have a small network of 4 machines one of which is the internet access box with a "do not speak unless spoken to" fire wall set up. ie firewall is set up so that only external incoming connections that are related to outgoing connections are allowed. NO machine is on all the time, so the ADSL line will mostly be connected to machine thats switched off. Lindsay

11 20