netdev - Slow OOM in netif

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20080124211810.3E24A46E9A@smtp.obs.bg>
Date:	Thu, 24 Jan 2008 23:18:10 +0200
From:	"Ivan H. Dichev" <idichev@....bg>
To:	netdev@...r.kernel.org
Subject: Slow OOM in netif_RX function

Eric Dumazet writes: 

> Ivan Dichev a écrit :
>> Hello,
>> I got problem with my linux router. It has slow persistent OOM
>> problems from few months ago.
>> Every working(I mean days when more traffic is generated) day my
>> router is leaking with 15-20 MB memory and
>> after 2 weeks the restart is a MUST.
>>> From /proc/slabinfo I saw that size-2048 and size-512 are growing
>> rapidly every day when traffic occur. 
>> 
>> --------- /proc/slabinfo --------------------
>> size-2048          20322  20349   2072    3    2 : tunables   24  12    0 
>> : slabdata   6780   6783      0
>> size-512           50984  51016    536    7    1 : tunables   32  16    0 
>> : slabdata   7288   7288      0 
>> 
>> 
>> I was wondering who is allocating this mem pools and then I changed
>> the kernel with 2.6.23-rc12 including  options
>> CONFIG_DEBUG_SLAB=y
>> CONFIG_DEBUG_SLAB_LEAK=y 
>> 
>> 
>> Unfortunately changing the kernel didn't solve the mem leak....
>> Now /proc/slab_allocators is showing that 3c59x driver is allocating
>> 2048 and 512 bytes memory pools
>> caused by RX function.
>> --------- from /proc/slab_allocators ------------------------------
>> 7612 size-2048: boomerang_rx+0x33b/0x437 [3c59x]
>> 16018 size-512: boomerang_rx+0x165/0x437 [3c59x] 
>> 
>> I was thinking that the 3com driver is bogus, .. but not!
>> After few days I changed the cards with rtl8139 and now ....
>> --------- from /proc/slab_allocators ------------------------------
>> size-2048: 20159 rtl8139_rx+0x155/0x2dc [8139too]
>> size-1024: 2693 rtl8139_rx+0x155/0x2dc [8139too]
>> size-512: 50515 rtl8139_rx+0x155/0x2dc [8139too] 
>> 
>> the memory leak appear again in the same function(RX). 
>> 
>> I did search over the mailing list and found as similar only this
>> http://www.spinics.net/lists/kernel/old/2003-q4/msg03071.html 
>> 
>> 
>> For sure it does not depend on kernel version and network
>> driver(except case if both drivers are bogus :)
>> Any ideas ? 
>> 
> 
> Could you post your iptable rules "iptables -t nat -nvL ; iptables -nvL", 
> and full "cat /proc/slabinfo" ? 
> 
> 
 

Sorry but my firewall is 7000+ lines and I cant paste the rules.
And that's why it's very hard to debug the chains :( 

I have better idea! 

What could happen if I put different Lan card in every slot?
In ex. to-private -> 3com
       to-inet    -> VIA
       to-dmz     -> rtl8139
And then to look which RX function is consuming the memory.
(boomerang_rx, rtl8139_rx, ... etc) 

With this it will be easier to understand which iptables rules(bound to the 
found interface) have to be watched.
(I am not sure that it will work ?) 

Any other ideas appreciated. 

Ivan Dichev
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html