Date:	Fri, 15 Jun 2012 15:10:15 +0600
From:	Mike Kazantsev <mk.fraggod@...il.com>
To:	netdev@...r.kernel.org
Subject: PROBLEM: regression in ipv6 / udpv6_sendmsg between 3.2/3.3
 kernels, causing panics


Good day,

Local IPv6-enabled system suffers from really frequent (but somewhat
random) kernel panics with mainline kernel versions starting from 3.3,
always featuring udpv6_sendmsg in the produced stack trace.

Machine in question acts as an internet gateway, hosting quite a few
services itself, traffic goes through it as well as originates from it.

It is connected to IPv6 via sit tunnel over regular ISP-provided direct
IPv4 connection (no tunneling, just dhcp-assigned internet IPv4).

First time I've seen this error was just a few seconds after network was
configured on first boot with 3.3 kernel.
I've tried updating kernel to later 3.3.X versions and 3.4 series
(stack trace captured is from 3.4.2).
Machine worked months with different 3.2.X kernels (starting from 3.2,
latest being 3.2.16) with no problems of this kind whatsoever (no major
kernel errors like oops, bug or panic).

Panic seem to happen reliably within 12 hours of booting kernel 3.3+ and
enabling ipv6 sit tunnel to the outside world.
So far, I've seen about 5-10 of these and they always feature
udpv6_sendmsg somewhere at the top.
Latest traceback (with 3.4.2, ver_linux output attached as well) on a
laptop camera and is attached.

I've tried reproducing it on local ipv6 network with iperf-generated
udp traffic (since there doesn't seem to be any local ipv6/udp services
on the machine) with no luck thus far.

In most cases, mainline (Linus tree) kernel versions from kernel.org
were used with AppArmor userspace compatibility patches
(http://wiki.apparmor.net/index.php/Main_Page#Userspace, included in the
archive).
Seeing apparmor calls in the stack traces (and since no apparmor
profiles are used on this machine), I've tried disabling AppArmor LSM,
but eventually got the same panic.

Hardware is a low-spec commodity amd64 Intel Atom miniITX platform.
Kernel config and hardware information attached.

Only workarounds I've been able to find so far is running 3.2.X kernels
or disabling sit tunnel (and IPv6 connectivity), but both seem to be
far from perfect.

Bisection doesn't seem to be an easy option, because error seem to be
fairly random, not triggered by just any udp traffic.

Can someone maybe point out specific commits I can try to revert, to
see if it'll solve the issue?
I haven't looked at the changes to the relevant code paths in 3.2-3.3
timeframe myself yet, but intend to try eventually, though I'm not
familiar enough with the kernel architecture and code to make much
sense of it, I'm afraid.

Thanks in advance for any assistance.


-- 
Mike Kazantsev // fraggod.net

View attachment "udpv6_sendmsg_cpuinfo.txt" of type "text/plain" (2936 bytes)

View attachment "udpv6_sendmsg_iomem.txt" of type "text/plain" (2171 bytes)

View attachment "udpv6_sendmsg_ioports.txt" of type "text/plain" (1565 bytes)

View attachment "udpv6_sendmsg_kconfig.txt" of type "text/plain" (73232 bytes)

View attachment "udpv6_sendmsg_lspci.txt" of type "text/plain" (25545 bytes)

View attachment "udpv6_sendmsg_message.txt" of type "text/plain" (2581 bytes)

View attachment "udpv6_sendmsg_modules.txt" of type "text/plain" (2748 bytes)

Download attachment "udpv6_sendmsg_panic.jpg" of type "image/jpeg" (93060 bytes)

View attachment "udpv6_sendmsg_ver_linux.txt" of type "text/plain" (1338 bytes)

Download attachment "signature.asc" of type "application/pgp-signature" (199 bytes)

Powered by blists - more mailing lists