linux-kernel - Re: Kernel falls apart under light memory pressure (i.e. linking vmlinux)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 17 May 2011 15:22:36 -0400
From:	Andrew Lutomirski <luto@....edu>
To:	Wu Fengguang <fengguang.wu@...el.com>
Cc:	Minchan Kim <minchan.kim@...il.com>,
	Andi Kleen <andi@...stfloor.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: Kernel falls apart under light memory pressure (i.e. linking vmlinux)

On Tue, May 17, 2011 at 2:00 AM, Wu Fengguang <fengguang.wu@...el.com> wrote:
> On Sun, May 15, 2011 at 12:12:36PM -0400, Andrew Lutomirski wrote:
>> On Sun, May 15, 2011 at 11:27 AM, Wu Fengguang <fengguang.wu@...el.com> wrote:
>>
>> That was probably because one of my testcases creates a 1.4GB file on
>> ramfs.  (I can provoke the problem without doing evil things like
>> that, but the test script is rather reliable at killing my system and
>> it works fine on my other machines.)
>
> Ah I didn't read your first email.. I'm now running
>
> ./test_mempressure.sh 1500 1400 1
>
> with mem=2G and no swap, but cannot reproduce OOM.

Do you have a Sandy Bridge laptop?  There was a recent thread on lkml
suggesting that only Sandy Bridge laptops saw this problem.  Although
there's something else needed to trigger it, because I can't do it
from an initramfs I made that tried to show this problem.

>
> What's your kconfig?

Attached.  This is 2.6.38.6.

>
>> If you want, I can try to generate a trace that isn't polluted with
>> the evil ramfs file.
>
> No, thanks. However it would be valuable if you can retry with this
> patch _alone_ (without the "if (need_resched()) return false;" change,
> as I don't see how it helps your case).
>
> @@ -2286,7 +2290,7 @@ static bool sleeping_prematurely(pg_data_t
> *pgdat, int order, long remaining,
>        * must be balanced
>        */
>       if (order)
> -               return pgdat_balanced(pgdat, balanced, classzone_idx);
> +               return !pgdat_balanced(pgdat, balanced, classzone_idx);
>       else
>               return !all_zones_ok;
>  }

Done.

I logged in, added swap, and ran a program that allocated 1900MB of
RAM and memset it.  The system lagged a bit but survived.  kswapd
showed 10% CPU (which is odd, IMO, since I'm using aesni-intel and I
think that all the crypt happens in kworker when aesni-intel is in
use).

Then I started Firefox, loaded gmail, and ran test_mempressure.sh.
Kaboom!  (I.e. system was hung)  SysRq-F saved the system and produced
the attached dump.  I had 6GB swap available, so there shouldn't have
been any OOM.

--Andy

Download attachment "messages.txt.xz" of type "application/x-xz" (15212 bytes)

Download attachment ".config" of type "application/octet-stream" (88497 bytes)