strange 6in4 errors - what does "RX errors" mean?
# ifconfig 6in4 6in4: flags=209<UP,POINTOPOINT,RUNNING,NOARP> mtu 1280 inet6 2a01:4f8:140:71f5:1::ffff prefixlen 128 scopeid 0x0<global> inet6 fe80::a08:1 prefixlen 64 scopeid 0x20<link> inet6 fe80::2e04:7ca5 prefixlen 64 scopeid 0x20<link> inet6 fe80::a0a:a01 prefixlen 64 scopeid 0x20<link> sit txqueuelen 1 (IPv6-in-IPv4) RX packets 563 bytes 69361 (67.7 KiB) RX errors 1200 dropped 0 overruns 0 frame 1200 TX packets 600 bytes 164726 (160.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 Above is the result of running ifconfig on my 6in4 device. The problem is the "RX errors" which correspond to ICMP packets missing. The system at the far end of the 6in4 link has monitoring that includes "ping6 www.google.com", until recently that was fine, since upgrading to Debian kernel 4.9.0-7-amd64 #1 SMP Debian 4.9.110-1 problems have started. The problems are somewhat intermittent. I don't expect that the recent Debian kernel upgrade broke this, maybe the reboot exposed some other problem. Does anyone know what "RX errors" means in the context of 6in4 interfaces on Linux? Is there any way of getting the kernel to log every error to dmesg or something? I expect this will be a really easy problem to solve once I know what the errors mean. When the errors happen I run tcpdump on the far end of the 6in4 link and see the packets going out, I run tcpdump on the ethernet device and see the encapsulated packets going in, and when I run tcpdump on the 6in4 device I don't see those packets but I do see packets for local ipv6 addresses. Here's the /etc/network/interfaces section from the server: auto 6in4 iface 6in4 inet6 v4tunnel address 2a01:4f8:140:71f5:1::ffff netmask 128 endpoint 203.15.121.80 ttl 64 up ip link set mtu 1280 dev 6in4 up ip route add 2a01:4f8:140:71f5:1::/96 dev 6in4 Here's the /etc/network/interfaces section from the other end: auto 6in4 iface 6in4 inet6 v4tunnel address 2a01:4f8:140:71f5:1::1 netmask 128 endpoint 46.4.124.165 ttl 64 up ip route add 2a01:4f8:140:71f5:1::ffff dev 6in4 up ip link set mtu 1280 dev 6in4 up ip route add 2000::/3 dev 6in4 Here's the result of running tcpdump on the server, those encapsulated packets look ok and should go to the 6in4 device. # tcpdump -i br0 -n host 203.15.121.80 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes 01:18:57.333752 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4 > 2404:6800:4006:800::2004: ICMP6, echo request, seq 14, length 64 01:18:58.357831 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4 > 2404:6800:4006:800::2004: ICMP6, echo request, seq 15, length 64 01:18:59.381702 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4 > 2404:6800:4006:800::2004: ICMP6, echo request, seq 16, length 64 01:19:00.405686 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4 > 2404:6800:4006:800::2004: ICMP6, echo request, seq 17, length 64 Any ideas? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
I've done some more research on this. From net/ipv6/ip6_tunnel.c in the kernel source the following is the only code that increments the frame error count: err = dscp_ecn_decapsulate(tunnel, ipv6h, skb); if (unlikely(err)) { if (log_ecn_err) net_info_ratelimited("non-ECT from %pI6 with DS=%#x\n", &ipv6h->saddr, ipv6_get_dsfield(ipv6h)); if (err > 1) { ++tunnel->dev->stats.rx_frame_errors; ++tunnel->dev->stats.rx_errors; goto drop; } } I believe that the below means that messages of level INFO (6) and below number (low number means high severity) will be logged by the kernel. # cat /proc/sys/kernel/printk 7 4 1 7 However the following grep returns no matches. # grep ECT.*with.DS /var/log/kern.log Any suggestions on what I should investigate next? On Wednesday, 25 July 2018 11:20:15 AM AEST Russell Coker wrote:
# ifconfig 6in4 6in4: flags=209<UP,POINTOPOINT,RUNNING,NOARP> mtu 1280 inet6 2a01:4f8:140:71f5:1::ffff prefixlen 128 scopeid 0x0<global> inet6 fe80::a08:1 prefixlen 64 scopeid 0x20<link> inet6 fe80::2e04:7ca5 prefixlen 64 scopeid 0x20<link> inet6 fe80::a0a:a01 prefixlen 64 scopeid 0x20<link> sit txqueuelen 1 (IPv6-in-IPv4) RX packets 563 bytes 69361 (67.7 KiB) RX errors 1200 dropped 0 overruns 0 frame 1200 TX packets 600 bytes 164726 (160.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Above is the result of running ifconfig on my 6in4 device. The problem is the "RX errors" which correspond to ICMP packets missing.
The system at the far end of the 6in4 link has monitoring that includes "ping6 www.google.com", until recently that was fine, since upgrading to Debian kernel 4.9.0-7-amd64 #1 SMP Debian 4.9.110-1 problems have started.
The problems are somewhat intermittent.
I don't expect that the recent Debian kernel upgrade broke this, maybe the reboot exposed some other problem.
Does anyone know what "RX errors" means in the context of 6in4 interfaces on Linux? Is there any way of getting the kernel to log every error to dmesg or something? I expect this will be a really easy problem to solve once I know what the errors mean.
When the errors happen I run tcpdump on the far end of the 6in4 link and see the packets going out, I run tcpdump on the ethernet device and see the encapsulated packets going in, and when I run tcpdump on the 6in4 device I don't see those packets but I do see packets for local ipv6 addresses.
Here's the /etc/network/interfaces section from the server:
auto 6in4 iface 6in4 inet6 v4tunnel address 2a01:4f8:140:71f5:1::ffff netmask 128 endpoint 203.15.121.80 ttl 64 up ip link set mtu 1280 dev 6in4 up ip route add 2a01:4f8:140:71f5:1::/96 dev 6in4
Here's the /etc/network/interfaces section from the other end:
auto 6in4 iface 6in4 inet6 v4tunnel address 2a01:4f8:140:71f5:1::1 netmask 128 endpoint 46.4.124.165 ttl 64 up ip route add 2a01:4f8:140:71f5:1::ffff dev 6in4 up ip link set mtu 1280 dev 6in4 up ip route add 2000::/3 dev 6in4
Here's the result of running tcpdump on the server, those encapsulated packets look ok and should go to the 6in4 device.
# tcpdump -i br0 -n host 203.15.121.80 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes 01:18:57.333752 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4
2404:6800:4006:800::2004: ICMP6, echo request, seq 14, length 64 01:18:58.357831 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4 2404:6800:4006:800::2004: ICMP6, echo request, seq 15, length 64 01:18:59.381702 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4 2404:6800:4006:800::2004: ICMP6, echo request, seq 16, length 64 01:19:00.405686 IP 203.15.121.80 > 46.4.124.165: IP6 2a01:4f8:140:71f5:1::4 2404:6800:4006:800::2004: ICMP6, echo request, seq 17, length 64
Any ideas?
-- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
participants (1)
-
Russell Coker