linux-kernel - Re: OOM-killer and strange RSS value in 3.9-rc7

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 2 May 2013 18:56:37 +0800
From:	Han Pingtian <hanpt@...ux.vnet.ibm.com>
To:	LKML <linux-kernel@...r.kernel.org>
Cc:	Michal Hocko <mhocko@...e.cz>, Christoph Lameter <cl@...ux.com>,
	penberg@...nel.org, rientjes@...gle.com, linux-mm@...ck.org
Subject: Re: OOM-killer and strange RSS value in 3.9-rc7

On Mon, Apr 29, 2013 at 04:57:11PM +0200, Michal Hocko wrote:
> On Mon 29-04-13 14:50:08, Christoph Lameter wrote:
> > On Sat, 27 Apr 2013, Han Pingtian wrote:
> > 
> > > and it is called so many times that the boot cannot be finished. So
> > > maybe the memory isn't freed even though __free_slab() get called?
> > 
> > Ok that suggests an issue with the page allocator then.
> 
> You seem to have CONFIG_MEMCG_KMEM enabled. Do you see the same issue
> when this is disabled? The kmem accounting should be disabled unless a
> specific limit is set but it would be better to know that this is not
> the factor.
> 
I have tested to disable CONFIG_MEMCG_KMEM. But it doesn't solve this
problem, I can still trigger the OOM by compiling kernel.

Now I suspect the problem comes from the driver "ibmvscsi". Because on another
power 7 system, which doesn't use ibmvscsi, there is no such OOM
problem here. For now, looks like only systems using ibmvscsi can
trigger this OOM problem. I have rebooted one of the ibmvscsi systems
with "init=/bin/sh" and compared the loaded modules with one of the
none-ibmvscsi system with "comm":

	$ comm -13 --check-order none-ibmvscsi.txt ibmvscsi.txt
	ibmvscsi
	nx_crypto
	scsi_transport_srp

the scsi_transport_srp is used by ibmvscsi and I can rmmod the 
nx_crypto out. Then I launched the compiling process on the
single-user-booted ibmvscsi system. The OOM can still be
produced on it.

Looks like "ibmvscsi" + "slub" can trigger this problem.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/