linux-kernel - Re: Kernel falls apart under light memory pressure (i.e. linking vmlinux)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 19 May 2011 10:16:50 -0400
From:	Andrew Lutomirski <luto@....edu>
To:	Minchan Kim <minchan.kim@...il.com>
Cc:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Wu Fengguang <fengguang.wu@...el.com>,
	Andi Kleen <andi@...stfloor.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Mel Gorman <mgorman@...e.de>,
	Johannes Weiner <hannes@...xchg.org>,
	Rik van Riel <riel@...hat.com>
Subject: Re: Kernel falls apart under light memory pressure (i.e. linking vmlinux)

I just booted 2.6.38.6 with exactly two patches applied.  Config was
the same as I emailed yesterday.  Userspace is F15.  First was
"aesni-intel: Merge with fpu.ko" because dracut fails to boot my
system without it.  Second was this (sorry for whitespace damage):

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 0665520..3f44b81 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -307,7 +307,7 @@ static void set_reclaim_mode(int priority, struct
scan_control *sc,
         */
        if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
                sc->reclaim_mode |= syncmode;
-       else if (sc->order && priority < DEF_PRIORITY - 2)
+       else if ((sc->order && priority < DEF_PRIORITY - 2) ||
priority <= DEF_PRIORITY / 3)
                sc->reclaim_mode |= syncmode;
        else
                sc->reclaim_mode = RECLAIM_MODE_SINGLE | RECLAIM_MODE_ASYNC;
@@ -1342,10 +1342,6 @@ static inline bool
should_reclaim_stall(unsigned long nr_taken,
        if (current_is_kswapd())
                return false;

-       /* Only stall on lumpy reclaim */
-       if (sc->reclaim_mode & RECLAIM_MODE_SINGLE)
-               return false;
-
        /* If we have relaimed everything on the isolated list, no stall */
        if (nr_freed == nr_taken)
                return false;

I started GNOME and Firefox, enabled swap, and ran test_mempressure.sh
1500 1400 1.  The system quickly gave the attached oops.

The oops was the ud2 here:

   0xffffffff810d251b <+215>:	mov    -0x28(%rbx),%rax
   0xffffffff810d251f <+219>:	test   $0x40,%al
   0xffffffff810d2521 <+221>:	je     0xffffffff810d2525 <shrink_page_list+225>
   0xffffffff810d2523 <+223>:	ud2

Please let me know what the next test to run is.

--Andy

Download attachment "IMG_20110519_094454.jpg" of type "image/jpeg" (96453 bytes)