netdev - Re: ixgbe NULLL pointer dereference on OOM condition, 2.6.31.7

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 12 Jan 2010 18:13:10 -0800 (Pacific Standard Time)
From:	"Brandeburg, Jesse" <jesse.brandeburg@...el.com>
To:	Ben Greear <greearb@...delatech.com>
cc:	NetDev <netdev@...r.kernel.org>, jesse.brandeburg@...el.com
Subject: Re: ixgbe NULLL pointer dereference on OOM condition, 2.6.31.7

On Mon, 4 Jan 2010, Ben Greear wrote:

> This is on a hacked 2.6.31.7 kernel.  I'm testing an application that creates
> 30,000+ TCP connections (to self).  The system is 64-bit with 12GB of RAM, but
> it can still run out of usable RAM (say, when I start another 10k connections
> to bring it up to 40k).
> 
> It looks like something in ixgbe isn't properly checking for inability
> to allocate (or to have previously allocated) an skb, or perhaps some other
> chunk of memory:
> 
> [root@...03-10G-09 ~]# BUG: unable to handle kernel NULL pointer dereference at 00000000000000e8
> IP: [<ffffffffa0054ca0>] ixgbe_clean_rx_irq+0xe4/0x522 [ixgbe]

Hi Ben, thanks for the report, is there a chance you can run gdb on your 
kernel (was it compiled with debug info?) and check the
gdb> l *(ixgbe_clean_rx_irq+0xe4)

Sorry I'm so slow to respond.

it seems there are some unwind problems after the recent round of patches 
to remove skb_dma_map/unmap, but those were only introduced in 2.6.33-rc1.  
Before that we weren't aware of any failure path issues.

I'm building a kernel now to see if I can figure out the offset where 
you're showing the problem.

Jesse
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html