linux-kernel - Re: OOM-Killer kills too much with 2.6.32.2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <2f11576a1001260503o5f987512n3c39ae1c3caa43f3@mail.gmail.com>
Date:	Tue, 26 Jan 2010 22:03:06 +0900
From:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
To:	Chris Wilson <chris@...is-wilson.co.uk>
Cc:	Roman Jarosz <kedgedev@...il.com>,
	lkml <linux-kernel@...r.kernel.org>,
	A Rojas <nqn1976list@...il.com>,
	Hugh Dickins <hugh.dickins@...cali.co.uk>,
	"A. Boulan" <arnaud.boulan@...ertysurf.fr>, michael@...nelt.co.at,
	jcnengel@...glemail.com, rientjes@...gle.com, earny@...4u.de,
	Jesse Barnes <jbarnes@...tuousgeek.org>,
	Eric Anholt <eric@...olt.net>
Subject: Re: OOM-Killer kills too much with 2.6.32.2

>> This comment is lie. __GFP_NORETY cause ENOMEM to shmem, not GEM itself.
>> GEM can't handle nor recover it. I suspect following commit is wrong.
>
> Indeed, the NORETRY flag is required for the inode mapping routines to
> return ENOMEM instead of triggering the OOM killer themselves. GEM has
> code to handle the ENOMEM as returned from shmem (please at least read the
> code before commenting, and comments are appreciated), by attempting to
> free up some of its own inactive buffers before retrying the allocation
> (with NORETRY removed, so the OOM killer will be invoked on the second
> instance). The reason for this convoluted approach is that GEM's inactive
> list shrinker requires the struct mutex and so cannot be run when GEM
> itself is attempting and failing to allocate memory. We could recover from
> more situations if we made some more invasive changes to our locking.
>
> This is without a doubt an area that needs improvement.

Please consider to revert such commit at once. Lots people reported
the same issue.
I really hope to stop bug report storm.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/