netdev - Re: PROBLEM: reproducible crash KVM+nf_conntrack all recent 2.6 kernels

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1264720891.2793.205.camel@tonnant>
Date:	Thu, 28 Jan 2010 18:21:31 -0500
From:	Jon Masters <jonathan@...masters.org>
To:	Patrick McHardy <kaber@...sh.net>
Cc:	linux-kernel <linux-kernel@...r.kernel.org>,
	netdev <netdev@...r.kernel.org>, netfilter-devel@...r.kernel.org
Subject: Re: PROBLEM: reproducible crash KVM+nf_conntrack all recent 2.6
 kernels

On Thu, 2010-01-28 at 13:19 +0100, Patrick McHardy wrote:
> ip6tables -t raw -I PREROUTING -j TRACE

Ok. Here are three crashes for you in a row, with the correct trace
options set and netfilter debug turned on this time. The latest config
used to reproduce this 100% reliably on 2.6.33-rc5 is attached.

The host system is configured as follows:

[jcm@...ihelion ~]$ /sbin/ifconfig
br0       Link encap:Ethernet  HWaddr 00:13:72:A0:60:F3  
          inet addr:192.168.1.3  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::213:72ff:fea0:60f3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:313 errors:0 dropped:0 overruns:0 frame:0
          TX packets:226 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:74337 (72.5 KiB)  TX bytes:43151 (42.1 KiB)

eth0      Link encap:Ethernet  HWaddr 00:13:72:A0:60:F3  
          inet6 addr: fe80::213:72ff:fea0:60f3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:299 errors:0 dropped:0 overruns:0 frame:0
          TX packets:229 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:78655 (76.8 KiB)  TX bytes:44277 (43.2 KiB)
          Interrupt:16 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:151 errors:0 dropped:0 overruns:0 frame:0
          TX packets:151 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:16409 (16.0 KiB)  TX bytes:16409 (16.0 KiB)

The KVM guests are configured within the 192.168.1 subnet:

kernel.bos.jonmasters.org. 192.168.1.10
fedora.bos.jonmasters.org. 192.168.1.210
rawhide.bos.jonmasters.org. 192.168.1.211

Exact sequence tested (useful mental note and record here):

1). Boot host system. Ensure that standard Fedora F12 out-of-the-box
firewall rules are activated, and iptables/ip6tables start up. This is
after Fedora added the three sysctls to turn off bridge filtering
to /etc/sysctl.conf, which it was thought "worked around" this.
2). Setup trace options for logging netfilter traversal and "echo 7
>/proc/sys/kernel/printk" to set appropriate kernel loglevel.
3). Start libvirtd (autostarts "Fedora-Rawhide-x86_64" and
"kernel-x86_64", which are test VM instances).
4). Login to "kernel" (F12) and do a "yum clean", "yum update
--skip-broken" to generate network traffic over the net. This is the
host that I do autobuilds of Linus' tree on for my twitter feed.
5). Wait a while for first two VMs to start up sshd.
6). Start a third "Fedora-x86_64" VM. Guess it doesn't matter which one,
but I always use this one in my tests to avoid hurting the images.
7). Panic every time, shortly after "port 4(vnet2) entering forwarding
state" in the log messages. It panic()s within about 1 minute.

Let me know what else I can do to help you track this down. I am not a
netfilter developer, but I can follow instructions and learn :) I wish I
had enough time in the day to go learn that code though.

Jon.


View attachment "config-nflogtracedebug" of type "text/x-mpsub" (77892 bytes)

View attachment "kvm_crash_trace_20100128_1800.txt" of type "text/plain" (50598 bytes)

View attachment "kvm_crash_trace_20100128_1805.txt" of type "text/plain" (54714 bytes)

View attachment "kvm_crash_trace_20100128_1810.txt" of type "text/plain" (54839 bytes)