
Firstly I'm moving this to luv-main because it's about serious Linux networking issues. On Sun, 21 Apr 2013, Tony Langdon <vk3jed@gmail.com> wrote:
About to rebuild the network here after moving house. One issue I'm contemplating is performance and redundancy. One part of the network can't (easily) be reached by Cat 5/6, running cables t that part of the house would be too messy at best. I have two ways I can bridge this gap - Powerline Ethernet adapters, which have worked extremely well in the past, or WiFi, using an access point in client bridge mode.
Now the powerline adapters do work extremely well, with a rated speed of 85 Mbps. I've never had an issue, except for the switchmode supply of one laptop, which trashed the link (took a bit of detective work that one!). The biggest weakness of these devices is that they can't be battery backed up. If the mains goes down, so do they.
http://etbe.coker.com.au/2010/08/04/clusters-dont-work/ I wrote the above blog post about my experience with clusters. Some of this applies to failover of links. One significant difference between a home network and a serious server network is that most of the functions of your home network don't matter much when you are asleep or away. Therefore a redundancy which involves you logging in as root and running a route command will work a lot better on a home network. Of course my experience is that having a sysadmin login as root and manually fail things over is better than any cluster software implementation I've seen, but that's another issue.
WiFi can be backed up, especially since a lot of the gear I have will happily run off a 12V battery, and some of the systems on the far end will be running off a battery backed DC supply. However, performance with the WiFi solution isn't as good.
Is there a way I can (easily and cheaply) arrange to run on the powerline devices by default and fail over to WiFi, if the power goes down?
How often does power go down anyway? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

Actually, it's because I intend to bring some of my radio services back up that I'm looking at failover options. These services run 24/7, and short outages happen several times per year here. Any connections that are up can be dropped, or in some cases put into odd states. Manual login isn't really going to cut it, because I may not be around when things go pear shaped, or I may be asleep. The only scenario that would work for a manual login is if a screwy power supply takes down the power line Ethernet, because I'll know about that. And if I'm going to manually login to fix things, it's even easier to do the obvious and swap over the cat 5 cable at one of the switches to achieve that result. :) On Sunday, April 21, 2013, Russell Coker wrote:
Firstly I'm moving this to luv-main because it's about serious Linux networking issues.
On Sun, 21 Apr 2013, Tony Langdon <vk3jed@gmail.com <javascript:;>> wrote:
About to rebuild the network here after moving house. One issue I'm contemplating is performance and redundancy. One part of the network can't (easily) be reached by Cat 5/6, running cables t that part of the house would be too messy at best. I have two ways I can bridge this gap - Powerline Ethernet adapters, which have worked extremely well in the past, or WiFi, using an access point in client bridge mode.
Now the powerline adapters do work extremely well, with a rated speed of 85 Mbps. I've never had an issue, except for the switchmode supply of one laptop, which trashed the link (took a bit of detective work that one!). The biggest weakness of these devices is that they can't be battery backed up. If the mains goes down, so do they.
http://etbe.coker.com.au/2010/08/04/clusters-dont-work/
I wrote the above blog post about my experience with clusters. Some of this applies to failover of links.
One significant difference between a home network and a serious server network is that most of the functions of your home network don't matter much when you are asleep or away. Therefore a redundancy which involves you logging in as root and running a route command will work a lot better on a home network. Of course my experience is that having a sysadmin login as root and manually fail things over is better than any cluster software implementation I've seen, but that's another issue.
WiFi can be backed up, especially since a lot of the gear I have will happily run off a 12V battery, and some of the systems on the far end will be running off a battery backed DC supply. However, performance with the WiFi solution isn't as good.
Is there a way I can (easily and cheaply) arrange to run on the powerline devices by default and fail over to WiFi, if the power goes down?
How often does power go down anyway?
-- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

Sorry for top posting. Is it just one computer that needs redundant links? If so, just having both up with wifi as a higher metric will work. If its more than one computer across the bridge, you need something at layer2 like STP or HSRP/VRRP On 22/04/2013, at 8:45, Tony Langdon <vk3jed@gmail.com> wrote:
Actually, it's because I intend to bring some of my radio services back up that I'm looking at failover options. These services run 24/7, and short outages happen several times per year here. Any connections that are up can be dropped, or in some cases put into odd states.
Manual login isn't really going to cut it, because I may not be around when things go pear shaped, or I may be asleep. The only scenario that would work for a manual login is if a screwy power supply takes down the power line Ethernet, because I'll know about that. And if I'm going to manually login to fix things, it's even easier to do the obvious and swap over the cat 5 cable at one of the switches to achieve that result. :)
On Sunday, April 21, 2013, Russell Coker wrote:
Firstly I'm moving this to luv-main because it's about serious Linux networking issues.
On Sun, 21 Apr 2013, Tony Langdon <vk3jed@gmail.com> wrote:
About to rebuild the network here after moving house. One issue I'm contemplating is performance and redundancy. One part of the network can't (easily) be reached by Cat 5/6, running cables t that part of the house would be too messy at best. I have two ways I can bridge this gap - Powerline Ethernet adapters, which have worked extremely well in the past, or WiFi, using an access point in client bridge mode.
Now the powerline adapters do work extremely well, with a rated speed of 85 Mbps. I've never had an issue, except for the switchmode supply of one laptop, which trashed the link (took a bit of detective work that one!). The biggest weakness of these devices is that they can't be battery backed up. If the mains goes down, so do they.
http://etbe.coker.com.au/2010/08/04/clusters-dont-work/
I wrote the above blog post about my experience with clusters. Some of this applies to failover of links.
One significant difference between a home network and a serious server network is that most of the functions of your home network don't matter much when you are asleep or away. Therefore a redundancy which involves you logging in as root and running a route command will work a lot better on a home network. Of course my experience is that having a sysadmin login as root and manually fail things over is better than any cluster software implementation I've seen, but that's another issue.
WiFi can be backed up, especially since a lot of the gear I have will happily run off a 12V battery, and some of the systems on the far end will be running off a battery backed DC supply. However, performance with the WiFi solution isn't as good.
Is there a way I can (easily and cheaply) arrange to run on the powerline devices by default and fail over to WiFi, if the power goes down?
How often does power go down anyway?
-- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
luv-main mailing list luv-main@luv.asn.au http://lists.luv.asn.au/listinfo/luv-main

On 22/04/13 9:15 AM, hannah commodore wrote:
Sorry for top posting.
Is it just one computer that needs redundant links? If so, just having both up with wifi as a higher metric will work.
If its more than one computer across the bridge, you need something at layer2 like STP or HSRP/VRRP It would be a couple, all of those needing the redundancy are Linux boxes, ranging from a Raspberry Pi to a netbook. Hmm, having both interfaces up is probably not an option, thanks to NAT and the need for port forwarding (wouldn't changing interfaces mean changing IPs?
Anyway, by the looks of it, I need some new hardware, whatever I do, whether that be a PtP 802.11n link or STP. It's not urgent, just a "nice to have" at this stage, the redundancy requirement is a while off. -- 73 de Tony VK3JED http://vkradio.com

On 22/04/2013, at 10:48, Tony Langdon <vk3jed@gmail.com> wrote:
Hmm, having both interfaces up is probably not an option, thanks to NAT and the need for port forwarding (wouldn't changing interfaces mean changing IPs?
it shouldn't. arp in Linux will respond to any IP assigned on the machine, not just that interface. configurable via sysctl of course. you could put the one IP on a loopback and none on the other interfaces, and it should still work with NAT as long as those interfaces are up, at least

On 22/04/13 11:50 AM, hannah commodore wrote:
On 22/04/2013, at 10:48, Tony Langdon <vk3jed@gmail.com> wrote:
Hmm, having both interfaces up is probably not an option, thanks to NAT and the need for port forwarding (wouldn't changing interfaces mean changing IPs? it shouldn't. arp in Linux will respond to any IP assigned on the machine, not just that interface. configurable via sysctl of course. you could put the one IP on a loopback and none on the other interfaces, and it should still work with NAT as long as those interfaces are up, at least
I think I've found a possible solution to my needs. I'll try running BOTH links at the same time. A closer analysis of the situation revealed that only one machine (a Windows box) needs the consistent bandwidth of the powerline adapters, but it also doesn't need network access when the power is off. The Linux devices that will be running 24x7 don't need much bandwidth. As long as the latency is reasonably consistent (something a good wifi signal can easily manage), then they will work well. So the Linux devices will be on the far end of the wifi link. In the event of an actual link failure (a low probability event), then manual cable swapping will get everything going again, but under normal conditions, the critical systems will be protected from those annoying brownouts and the like.
_______________________________________________ luv-main mailing list luv-main@luv.asn.au http://lists.luv.asn.au/listinfo/luv-main
-- 73 de Tony VK3JED http://vkradio.com

Mostly replying via the Russell's post to the OP...
On Sun, 21 Apr 2013, Tony Langdon <vk3jed@gmail.com> wrote:
About to rebuild the network here after moving house. One issue I'm contemplating is performance and redundancy. One part of the network can't (easily) be reached by Cat 5/6, running cables t that part of the house would be too messy at best. I have two ways I can bridge this gap - Powerline Ethernet adapters, which have worked extremely well in the past, or WiFi, using an access point in client bridge mode.
Now the powerline adapters do work extremely well, with a rated speed of 85 Mbps. I've never had an issue, except for the switchmode supply of one laptop, which trashed the link (took a bit of detective work that one!). The biggest weakness of these devices is that they can't be battery backed up. If the mains goes down, so do they.
http://etbe.coker.com.au/2010/08/04/clusters-dont-work/
I wrote the above blog post about my experience with clusters. Some of this applies to failover of links.
One significant difference between a home network and a serious server network is that most of the functions of your home network don't matter much when you are asleep or away. Therefore a redundancy which involves you logging in as root and running a route command will work a lot better on a home network. Of course my experience is that having a sysadmin login as root and manually fail things over is better than any cluster software implementation I've seen, but that's another issue.
WiFi can be backed up, especially since a lot of the gear I have will happily run off a 12V battery, and some of the systems on the far end will be running off a battery backed DC supply. However, performance with the WiFi solution isn't as good.
A point to point 802.11n link should easily beat any powerline adapter, unless you have a solid lead wall or something?
Is there a way I can (easily and cheaply) arrange to run on the powerline devices by default and fail over to WiFi, if the power goes down?
If you know a bit about OSPF (or RIP) and can arrange a router at each end then yes, easily. If you have a proper wireless bridge at each end and can run Linux bridge at each end to bridge the two interfaces (wifi and powerline) and turn on STP then also yes, easily. My laptop has "LAN/WLAN switching" so that if I plug a network cable in the wireless switches off, which saves a heap of DHCP addresses when you consider 100 laptops on a single network. If your far end machine has that sort of capability then it would work in the "Ethernet link down" case, but not if there was some other fault that still presented a link connected without actually having end to end connectivity. If your far end machine runs linux then a simple ping script to swap over might be easiest. If it were me, I'd get a couple of openwrt based devices (netgear wndr3800's are what I've been using lately) and create separate /30 networks over each of the wireless and powerline links, then use OSPF to route over whichever one has connectivity, giving the powerline link priority if that has the better performance. Of course that doesn't work if you need the same broadcast domain, in which case I'd bridge them and use STP, but I'm less familiar with that. Also, that doesn't necessarily handle the case where powerline link is up but performing badly etc. James

On 22/04/13 10:20 AM, James Harper wrote:
Mostly replying via the Russell's post to the OP...
If you know a bit about OSPF (or RIP) and can arrange a router at each end then yes, easily. That's likely to cause other issues, since broadcasts won't propagate across the network (good way to screw up applications that rely on broadcasts). I know there are a couple that would need to operate across the link.
If you have a proper wireless bridge at each end and can run Linux bridge at each end to bridge the two interfaces (wifi and powerline) and turn on STP then also yes, easily. That's one for the future
My laptop has "LAN/WLAN switching" so that if I plug a network cable in the wireless switches off, which saves a heap of DHCP addresses when you consider 100 laptops on a single network. If your far end machine has that sort of capability then it would work in the "Ethernet link down" case, but not if there was some other fault that still presented a link connected without actually having end to end connectivity. If your far end machine runs linux then a simple ping script to swap over might be easiest. I think the Ethernet link stays up if the powerline connectivity is lost. The rest would only work is there was a Linux box directly connected to both interfaces (there won't be, there will be a switch).
If it were me, I'd get a couple of openwrt based devices (netgear wndr3800's are what I've been using lately) and create separate /30 networks over each of the wireless and powerline links, then use OSPF to route over whichever one has connectivity, giving the powerline link priority if that has the better performance. Of As above, not the ideal case, there are a couple of applications used across the network occasionally that rely on being in the same broadcast domain (Bonjour based, etc). course that doesn't work if you need the same broadcast domain, in which case I'd bridge them and use STP, but I'm less familiar with that. Also, that doesn't necessarily handle the case where powerline link is up but performing badly etc. I've only had all or nothing performance in the past, but I don't think there is any hardware indication of link failure (i.e. the connected device still sees a link active), it simply looks the same as if you were plugged into a dumb switch with nothing else connected to it.
-- 73 de Tony VK3JED http://vkradio.com

Russell Coker <russell@coker.com.au> writes:
One significant difference between a home network and a serious server network is that most of the functions of your home network don't matter much when you are asleep or away. Therefore a redundancy which involves you logging in as root and running a route command will work a lot better on a home network. Of course my experience is that having a sysadmin login as root and manually fail things over is better than any cluster software implementation I've seen, but that's another issue.
+1. This is the approach I have taken for SMEs as well -- I usually try to simplify it so that there is a (metaphorical or actual) throw switch, so that if it goes tits-up the customer can flip it and hopefully things will be at least usable until a trained sysadmin can clean up the mess properly. Trying to teach the computer when to flip it tends to be nontrivial and tends to introduce new SPOFs. I *want* to believe things are handled better in the Enterprise! space, but my personal experience is negligiible there and I'm a pessimist.

Russell Coker <russell@coker.com.au> writes:
One significant difference between a home network and a serious server network is that most of the functions of your home network don't matter much when you are asleep or away. Therefore a redundancy which involves you logging in as root and running a route command will work a lot better on a home network. Of course my experience is that having a sysadmin login as root and manually fail things over is better than any cluster software implementation I've seen, but that's another issue.
+1. This is the approach I have taken for SMEs as well -- I usually try to simplify it so that there is a (metaphorical or actual) throw switch, so that if it goes tits-up the customer can flip it and hopefully things will be at least usable until a trained sysadmin can clean up the mess properly. Trying to teach the computer when to flip it tends to be nontrivial and tends to introduce new SPOFs.
I *want* to believe things are handled better in the Enterprise! space, but my personal experience is negligiible there and I'm a pessimist.
The problem is that there are so many modes of failure, and when you've accounted for them all, murphy will invent new ones. Even RAID (especially software RAID) often requires user intervention when disks play up but don't outright die. We recently had a customer server (HP ML350 with HP hardware RAID) play up, running extremely slowly for roughly 2 minutes out of every 3. We suspected a bad disk, but _nothing_ in any logs or other measurable thing indicated which disk, and pulling the potentially wrong disk out of a RAID5 is not something to be considered lightly. In the end the onsite manager noted that disk 1 would flash strangely on occasion, and those occasions lined up exactly with the freezing, so after confirming that the most recent backups were successful and making sure the customer understood the consequences we pulled the oddly behaving disk, and the effect on performance was positive and instant. A new disk failed instantly in that slot, but worked in another slot. Replacing the backplane and reinserting the original disk brought the problem back with no indication of actual failure. Only with a new disk and new backplane were things good again. My best guess was that there was some sort of failure on the original disk such that inserting it corrupted the slot in the backplane until a reboot, but that's a guess and having solved the problem I didn't much care anymore as the hardware is covered by service contract. The biggest problem you'll have with any sort of simple failover arrangement is flapping, where something has failed such that it works for 5 seconds then doesn't work for 5 seconds, for various values of "works" and "doesn't work". I've seen datacentre's fall victim to this before - a redundant link doesn't help much if the primary link comes up and down every few seconds and the failover keeps swapping back to the original link just as it fails again. So even "Enterprise" can fall victim to such things. Fortunately such problems are relatively rare and STP/OSPF will be all that is required most of the time, with someone needing to pull the plug manually on rare occasions. James
participants (5)
-
hannah commodore
-
James Harper
-
Russell Coker
-
Tony Langdon
-
trentbuck@gmail.com