16 Apr
2014
16 Apr
'14
2:24 a.m.
Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
The problem is that path MTU detection seem to break when the "bad"
modems are involved. So the Linux box running pppoe is OK, because it
knows the interface has an mtu+mru of 1492, but masqueraded clients do
not.
You can work around the problem a bit, by having an iptables rule with
--clamp-mss-to-pmtu, but it's a kludge.. and importantly, only
required for two of these four modems. The other two work just fine
*with apparently identical configurations* (ie. LLC / bridged)
Can anyone think of a reason for this?
The rp-pppoe and other mailing lists offer tantalising hints that I'm
not alone, but sadly those threads do not lead to any solutions.
Modems that work:
* TP-Link TD-8817 (Trendchip chipset)
* Billion 7300RA (Trendchip chipset)
Modems that don't work:
* TP-Link TD-8840T (Trendchip)
* Billion 7800NL (Broadcom chipset)
16 Apr
16 Apr
3 a.m.
Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
The problem is that path MTU detection seem to break when the "bad"
modems are involved. So the Linux box running pppoe is OK, because it
knows the interface has an mtu+mru of 1492, but masqueraded clients do
not.
You can work around the problem a bit, by having an iptables rule with
--clamp-mss-to-pmtu, but it's a kludge.. and importantly, only
required for two of these four modems. The other two work just fine
*with apparently identical configurations* (ie. LLC / bridged)
Can anyone think of a reason for this?
The rp-pppoe and other mailing lists offer tantalising hints that I'm
not alone, but sadly those threads do not lead to any solutions.
Modems that work:
* TP-Link TD-8817 (Trendchip chipset)
* Billion 7300RA (Trendchip chipset)
Modems that don't work:
* TP-Link TD-8840T (Trendchip)
* Billion 7800NL (Broadcom chipset)
In which direction "doesn't work for masq clients"?
For sending (client -> internet), the ICMP Fragmentation Required packet should get
sent from the ppp endpoint (eg your linux router). Pinging with 'do not fragment'
packets from your masq clients should confirm that this is working.
For receiving (internet -> client), something upstream of the LT2P endpoint has to send
the ICMP Frag Required packet to the sender of the packet. It can't be (directly) the
fault of your modem.
What could upset things though is if the ppp mtu and mru settings aren't getting
passed to the other end. So if your end has an mtu of 1492, your router won't allow
larger packets through and should (easily testable) send the frag required packet back,
but if the other end didn't know that your mtu was set thus, because you hadn't
specified an mru and/or because (maybe) the modem wasn't doing some magic somewhere to
let the other end know what sized packet you could receive, then you would have problems.
Are you definitely setting both mtu and mru in your ppp config?
Can you confirm that your router is behaving correctly for your clients? Eg on your
clients do "ping -M do -c 1 -s 1472 8.8.8.8". Then "ping -M do -c 1 -s 1464
8.8.8.8". The first should give you a fragmentation required response. The second
should not (don't know if google dns answers pings.
Now can do you the same from an external IP address (with 1500 MTU - not another PPPoE
endpoint)? Email me with your external IP privately and I can test this for you if you
don't have such access.
The other possibility is that somewhere between you and the LNS is an even lower MTU
restriction, so you'd need to set your mtu and mru to something lower. I would expect
you'd have problems with the host in that case though.
One further possibility is that even in bridge mode, some of these modems are snooping
your packets and setting the MSS for you. That would be easy enough to test too by making
a connection to something running wireshark. (can help you with that too if you want)
All that said though, you still do need to set the MSS. There are too many broken routers
out there.
James
4:27 a.m.
On 16 April 2014 13:00, James Harper <james.harper(a)bendigoit.com.au> wrote:
Hi James,
I am explicitly setting mtu and mru to 1492 in the pppd config.
I don't have any of the faulty modems connected right now, so can't
check until tonight -- but last time I brought this up on list, I
checked and said "If I'm reading the tcpdump correctly (see reply to
myself with it this morning) then yes, there is a frag-required
response." I will re-confirm this tonight.
So outbound (masq clients to internet via linux router) appeared to be
doing the right thing.
Last time this came up, in 2013, you checked the inbound packets, and reported:
"I can confirm what we thought - pings <= 1492 bytes get a response,
pings > 1492 bytes get no response, not even a 'fragmentation
required'."
The issue that confuses me deeply is that half the modems work, and
half don't - yet they have similar internals, and are configured as
identically as they can be (given differing user interfaces).
They are all setup to use LLC, an 8/35 VPI/VCI and to bridge PPPoE in
full-bridge mode.
I think that fact rules out my ISP or there being a dodgy router
somewhere beyond the ISP, as surely that would affect me regardless of
which modem is bridging?
I was willing to write off one modem as having mysteriously broken
firmware, but it seems unlikely that two modems from different vendors
would be broken this way -- and also likely that I'll continue to hit
the problem if I buy more modems :(
tjc
--
Turning and turning in the widening gyre
The falcon cannot hear the falconer
Things fall apart; the center cannot hold
Mere anarchy is loosed upon the world
Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
The problem is that path MTU detection seem to break when the "bad"
modems are involved. So the Linux box running pppoe is OK, because it
knows the interface has an mtu+mru of 1492, but masqueraded clients do
not.
You can work around the problem a bit, by having an iptables rule with
--clamp-mss-to-pmtu, but it's a kludge.. and importantly, only
required for two of these four modems. The other two work just fine
*with apparently identical configurations* (ie. LLC / bridged)
Can anyone think of a reason for this?
The rp-pppoe and other mailing lists offer tantalising hints that I'm
not alone, but sadly those threads do not lead to any solutions.
Modems that work:
* TP-Link TD-8817 (Trendchip chipset)
* Billion 7300RA (Trendchip chipset)
Modems that don't work:
* TP-Link TD-8840T (Trendchip)
* Billion 7800NL (Broadcom chipset)
In which direction "doesn't work for masq clients"?
For sending (client -> internet), the ICMP Fragmentation Required packet should get
sent from the ppp endpoint (eg your linux router). Pinging with 'do not fragment'
packets from your masq clients should confirm that this is working.
For receiving (internet -> client), something upstream of the LT2P endpoint has to
send the ICMP Frag Required packet to the sender of the packet. It can't be (directly)
the fault of your modem.
What could upset things though is if the ppp mtu and mru settings aren't getting
passed to the other end. So if your end has an mtu of 1492, your router won't allow
larger packets through and should (easily testable) send the frag required packet back,
but if the other end didn't know that your mtu was set thus, because you hadn't
specified an mru and/or because (maybe) the modem wasn't doing some magic somewhere to
let the other end know what sized packet you could receive, then you would have problems.
Are you definitely setting both mtu and mru in your ppp config?
Can you confirm that your router is behaving correctly for your clients? Eg on your
clients do "ping -M do -c 1 -s 1472 8.8.8.8". Then "ping -M do -c 1 -s 1464
8.8.8.8". The first should give you a fragmentation required response. The second
should not (don't know if google dns answers pings.
Now can do you the same from an external IP address (with 1500 MTU - not another PPPoE
endpoint)? Email me with your external IP privately and I can test this for you if you
don't have such access.
The other possibility is that somewhere between you and the LNS is an even lower MTU
restriction, so you'd need to set your mtu and mru to something lower. I would expect
you'd have problems with the host in that case though.
One further possibility is that even in bridge mode, some of these modems are snooping
your packets and setting the MSS for you. That would be easy enough to test too by making
a connection to something running wireshark. (can help you with that too if you want)
All that said though, you still do need to set the MSS. There are too many broken routers
out there.
James
4:42 a.m.
Hi James,
I am explicitly setting mtu and mru to 1492 in the pppd config.
I don't have any of the faulty modems connected right now, so can't
check until tonight -- but last time I brought this up on list, I
checked and said "If I'm reading the tcpdump correctly (see reply to
myself with it this morning) then yes, there is a frag-required
response." I will re-confirm this tonight.
So outbound (masq clients to internet via linux router) appeared to be
doing the right thing.
Last time this came up, in 2013, you checked the inbound packets, and
reported:
"I can confirm what we thought - pings <= 1492 bytes get a response,
pings > 1492 bytes get no response, not even a 'fragmentation
required'."
Yep. I remember now!
The issue that confuses me deeply is that half the
modems work, and
half don't - yet they have similar internals, and are configured as
identically as they can be (given differing user interfaces).
They are all setup to use LLC, an 8/35 VPI/VCI and to bridge PPPoE in
full-bridge mode.
I think that fact rules out my ISP or there being a dodgy router
somewhere beyond the ISP, as surely that would affect me regardless of
which modem is bridging?
I was willing to write off one modem as having mysteriously broken
firmware, but it seems unlikely that two modems from different vendors
would be broken this way -- and also likely that I'll continue to hit
the problem if I buy more modems :(
In the absence of anything else, I'm thinking that your modem is doing deep inspection
on the PPPoE packets and is setting the MSS on the encapsulated TCP packets according to
the current PPP parameters. In that case the "bad" modems aren't doing this
(I would call them "good" modems because I don't want something to screw
with my packets, but that's just me :)
To find out all you'd have to do is telnet to somewhere where you are running tcpdump
and see what is captured depending on what modem you are using.
James
5:19 a.m.
On 16 April 2014 14:42, James Harper <james.harper(a)bendigoit.com.au> wrote:
I guess that's possible, but it seems so unlikely to me.. These are
just consumer-grade ADSL modems, and not even particularly high-end
ones at that. Reconstructing the tcp streams inside the pppoe streams
inbetween the ISP and the linux server and then putting it all back
together sounds beyond their means to me.
But yeah.. it's possible.
Hi James,
I am explicitly setting mtu and mru to 1492 in the pppd config.
I don't have any of the faulty modems connected right now, so can't
check until tonight -- but last time I brought this up on list, I
checked and said "If I'm reading the tcpdump correctly (see reply to
myself with it this morning) then yes, there is a frag-required
response." I will re-confirm this tonight.
So outbound (masq clients to internet via linux router) appeared to be
doing the right thing.
Last time this came up, in 2013, you checked the inbound packets, and
reported:
"I can confirm what we thought - pings <= 1492 bytes get a response,
pings > 1492 bytes get no response, not even a 'fragmentation
required'."
Yep. I remember now!
The issue that confuses me deeply is that half
the modems work, and
half don't - yet they have similar internals, and are configured as
identically as they can be (given differing user interfaces).
They are all setup to use LLC, an 8/35 VPI/VCI and to bridge PPPoE in
full-bridge mode.
I think that fact rules out my ISP or there being a dodgy router
somewhere beyond the ISP, as surely that would affect me regardless of
which modem is bridging?
I was willing to write off one modem as having mysteriously broken
firmware, but it seems unlikely that two modems from different vendors
would be broken this way -- and also likely that I'll continue to hit
the problem if I buy more modems :(
In the absence of anything else, I'm thinking that your modem is doing deep
inspection on the PPPoE packets and is setting the MSS on the encapsulated TCP packets
according to the current PPP parameters. In that case the "bad" modems
aren't doing this (I would call them "good" modems because I don't want
something to screw with my packets, but that's just me :) To find out all you'd have to do is telnet to
somewhere where you are running tcpdump and see what is captured depending on what modem
you are using.
I'll investigate tonight.
5:23 a.m.
In the absence
of anything else, I'm thinking that your modem is doing
deep inspection on the PPPoE packets and is setting the MSS on the
encapsulated TCP packets according to the current PPP parameters. In that
case the "bad" modems aren't doing this (I would call them "good"
modems
because I don't want something to screw with my packets, but that's just me
:)
I guess that's possible, but it seems so unlikely to me.. These are
just consumer-grade ADSL modems, and not even particularly high-end
ones at that. Reconstructing the tcp streams inside the pppoe streams
inbetween the ISP and the linux server and then putting it all back
together sounds beyond their means to me.
11:07 a.m.
On 16/04/14 12:24, Toby Corkindale wrote:
Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
The problem is that path MTU detection seem to break when the "bad"
modems are involved. So the Linux box running pppoe is OK, because it
knows the interface has an mtu+mru of 1492, but masqueraded clients do
not.
You can work around the problem a bit, by having an iptables rule with
--clamp-mss-to-pmtu, but it's a kludge.. and importantly, only
required for two of these four modems. The other two work just fine
*with apparently identical configurations* (ie. LLC / bridged)
Can anyone think of a reason for this?
Wild guess - segmentation offload. I have seen offload do really strange
things with masqueraded packets and it is just possible that the bad
modems support offload but the good modems do not. On the linux box, issue
for interface in eth0
do
for option in tso ufo gso gro lro
do
ethtool $interface $option off
done
done
Replace eth0 with all the physical network interface names (eth0 eth1 etc.).
11:08 a.m.
On 16/04/14 21:07, Keith Owens wrote:
On 16/04/14 12:24, Toby Corkindale wrote:
<idiot>
for interface in eth0
do
for option in tso ufo gso gro lro
do
ethtool -K $interface $option off
done
done
</idiot>
Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
The problem is that path MTU detection seem to break when the "bad"
modems are involved. So the Linux box running pppoe is OK, because it
knows the interface has an mtu+mru of 1492, but masqueraded clients do
not.
You can work around the problem a bit, by having an iptables rule with
--clamp-mss-to-pmtu, but it's a kludge.. and importantly, only
required for two of these four modems. The other two work just fine
*with apparently identical configurations* (ie. LLC / bridged)
Can anyone think of a reason for this?
Wild guess - segmentation offload. I have seen offload do really strange
things with masqueraded packets and it is just possible that the bad
modems support offload but the good modems do not. On the linux box, issue
for interface in eth0
do
for option in tso ufo gso gro lro
do
ethtool $interface $option off
done
done
Replace eth0 with all the physical network interface names (eth0 eth1 etc.).
_______________________________________________
luv-main mailing list
luv-main(a)luv.asn.au
http://lists.luv.asn.au/listinfo/luv-main
17 Apr
17 Apr
1:48 a.m.
On 16 April 2014 21:07, Keith Owens <kaos(a)ocs.com.au> wrote:
On 16/04/14 12:24, Toby Corkindale wrote:
I don't have root shell access on the modems, so I can't run it there.
I experimented with adjusting ethernet options last time this came up
in 2013, to no effect.
But thanks for the suggestion.
Toby
Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
The problem is that path MTU detection seem to break when the "bad"
modems are involved. So the Linux box running pppoe is OK, because it
knows the interface has an mtu+mru of 1492, but masqueraded clients do
not.
You can work around the problem a bit, by having an iptables rule with
--clamp-mss-to-pmtu, but it's a kludge.. and importantly, only
required for two of these four modems. The other two work just fine
*with apparently identical configurations* (ie. LLC / bridged)
Can anyone think of a reason for this?
Wild guess - segmentation offload. I have seen offload do really strange
things with masqueraded packets and it is just possible that the bad
modems support offload but the good modems do not. On the linux box, issue
for interface in eth0
do
for option in tso ufo gso gro lro
do
ethtool $interface $option off
done
done
Replace eth0 with all the physical network interface names (eth0 eth1 etc.).
10:45 a.m.
On 17/04/14 11:48, Toby Corkindale wrote:
On 16 April 2014 21:07, Keith Owens
<kaos(a)ocs.com.au> wrote:
Not on the modems, issue the commands on the Linux box. Offload is a
negotiated attribute and if one end does not request offload, neither
end will do it.
On 16/04/14 12:24, Toby Corkindale wrote:
I
don't have root shell access on the modems, so I can't run it there.
I experimented with adjusting ethernet options last time this came up
in 2013, to no effect.
But thanks for the suggestion.
Toby Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
The problem is that path MTU detection seem to break when the "bad"
modems are involved. So the Linux box running pppoe is OK, because it
knows the interface has an mtu+mru of 1492, but masqueraded clients do
not.
You can work around the problem a bit, by having an iptables rule with
--clamp-mss-to-pmtu, but it's a kludge.. and importantly, only
required for two of these four modems. The other two work just fine
*with apparently identical configurations* (ie. LLC / bridged)
Can anyone think of a reason for this?
Wild guess - segmentation offload. I have seen offload do really strange
things with masqueraded packets and it is just possible that the bad
modems support offload but the good modems do not. On the linux box, issue
for interface in eth0
do
for option in tso ufo gso gro lro
do
ethtool $interface $option off
done
done
Replace eth0 with all the physical network interface names (eth0 eth1 etc.).
11:04 a.m.
Not on the modems, issue the commands on the Linux box. Offload is a
negotiated attribute and if one end does not request offload, neither
end will do it.
That is very incorrect.
All of the options you listed are internal to the network adapter itself and allow various
optimisations in how the OS sends and receives data from the network adapter. None of
those options changes the behaviour of the data on the wire, nor are they negotiated
between anything else on the network (there are some options that are, like flow control,
duplex etc, but these are not those)
If something was broken about the way the network adapter or driver implemented those
offloads (quite a common thing in the past) then it would be equally broken no matter what
modem was used, and would manifest itself in other ways.
James
29 Apr
29 Apr
4:14 a.m.
Hi,
I posted last year about a problem I was having with Linux's PPPoE
functionality in regards to a specific modem. At the time I put it
down to a dodgy modem and moved on, but now I've hit it on another
modem, and twice seems more than coincidence.
You aren't a Telstra customer are you?
http://www.reddit.com/r/australia/comments/2485wh/telstra_internet_traffic_…
I don't know why it would affect only some modem types though.
James
2988
days inactive
3001
days old
11 comments
3 participants
participants (3)
-
James Harper -
Keith Owens -
Toby Corkindale