linux-kernel - Re: Kernel falls apart under light memory pressure (i.e. linking vmlinux)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 23 May 2011 21:19:42 -0400
From:	Andrew Lutomirski <luto@....edu>
To:	Minchan Kim <minchan.kim@...il.com>
Cc:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	fengguang.wu@...el.com, andi@...stfloor.org, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org, mgorman@...e.de, hannes@...xchg.org,
	riel@...hat.com
Subject: Re: Kernel falls apart under light memory pressure (i.e. linking vmlinux)

On Sun, May 22, 2011 at 7:12 PM, Minchan Kim <minchan.kim@...il.com> wrote:
> Could you test below patch based on vanilla 2.6.38.6?
> The expect result is that system hang never should happen.
> I hope this is last test about hang.
>
> Thanks.
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 292582c..1663d24 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -231,8 +231,11 @@ unsigned long shrink_slab(struct shrink_control *shrink,
>       if (scanned == 0)
>               scanned = SWAP_CLUSTER_MAX;
>
> -       if (!down_read_trylock(&shrinker_rwsem))
> -               return 1;       /* Assume we'll be able to shrink next time */
> +       if (!down_read_trylock(&shrinker_rwsem)) {
> +               /* Assume we'll be able to shrink next time */
> +               ret = 1;
> +               goto out;
> +       }
>
>       list_for_each_entry(shrinker, &shrinker_list, list) {
>               unsigned long long delta;
> @@ -286,6 +289,8 @@ unsigned long shrink_slab(struct shrink_control *shrink,
>               shrinker->nr += total_scan;
>       }
>       up_read(&shrinker_rwsem);
> +out:
> +       cond_resched();
>       return ret;
>  }
>
> @@ -2331,7 +2336,7 @@ static bool sleeping_prematurely(pg_data_t
> *pgdat, int order, long remaining,
>        * must be balanced
>        */
>       if (order)
> -               return pgdat_balanced(pgdat, balanced, classzone_idx);
> +               return !pgdat_balanced(pgdat, balanced, classzone_idx);
>       else
>               return !all_zones_ok;
>  }

So far with this patch I can't reproduce the hang or the bogus OOM.

To be completely clear, I have COMPACTION, MIGRATION, and THP off, I'm
running 2.6.38.6, and I have exactly two patches applied.  One is the
attached patch and the other is a the fpu.ko/aesni_intel.ko merger
which I need to get dracut to boot my box.

For fun, I also upgraded to 8GB of RAM and it still works.

--Andy

>
> --
> Kind regards,
> Minchan Kim
>

Download attachment "minchan-patch-v3.patch" of type "application/octet-stream" (1215 bytes)