linux-kernel - Re: PROBLEM: reproducible crash KVM+nf

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1264712754.2793.154.camel@tonnant>
Date:	Thu, 28 Jan 2010 16:05:54 -0500
From:	Jon Masters <jonathan@...masters.org>
To:	Patrick McHardy <kaber@...sh.net>
Cc:	linux-kernel <linux-kernel@...r.kernel.org>,
	netdev <netdev@...r.kernel.org>, netfilter-devel@...r.kernel.org
Subject: Re: PROBLEM: reproducible crash KVM+nf_conntrack all recent 2.6
 kernels

On Thu, 2010-01-28 at 13:19 +0100, Patrick McHardy wrote:
> Jon Masters wrote:
> > On Thu, 2010-01-28 at 00:46 -0500, Jon Masters wrote:
> > 
> >> A number of people seem to have reported this crash in various forms,
> >> but I have yet to see a solution, and can reproduce on 2.6.33-rc5 this
> >> evening so I know it's still present in the latest upstream kernels too.
> >> Userspace is Fedora 12, and this happens on both all recent F12 kernels
> >> (sporadic in 2.6.31 until recently, solidly reproducible on 2.6.32) and
> >> upstream 2.6.32, and 2.6.33-rc5 also - hard to find a "known good".
> > 
> > Now I can capture the panic()s, I'm rebuilding the 2.6.33-rc5 kernel
> > with some better debugging options to at least get some more data.
> 
> Could you try "ip6tables -t raw -I PREROUTING -j TRACE" after loading
> the ip6t_LOG module? That way we can hopefully see the entire packet
> paths through netfilter.

Doing that now. Turning on a bunch of debug options like netfilter debug
and kmemleak causes it to trigger a lot less often, so I've reverted to
the "known bad" config that triggers this reliably.

Turning on netfilter debug and kmemleak resulted in a large number of:

[18493.405393] nf_conntrack: table full, dropping packet.
[18503.165713] kmemleak: 2 new suspected memory leaks
(see /sys/kernel/debug/kme
mleak)

etc. The memleak warnings don't look all that useful (attached what was
left in the buffer anyway) but the slowdown (KSM and kmemleak both at
near 100% CPU) seems to affect the timing of this and make it less
likely to trigger. The box fell over as I was rebooting it after a few
hours, and just after I detached the serial console to reset it :(

Attached current config. Send you the netfilter trace shortly.

Jon.

View attachment "config-known-to-trigger-minimal.txt" of type "text/plain" (77875 bytes)

View attachment "kmemleak_20100128_1" of type "text/plain" (64215 bytes)