linux-kernel - Re: Deadlock possibly caused by too_many

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <AANLkTimQzT3SYf0LMqEH8Bfta2gnvVjWCHRj=kmb_4Z7@mail.gmail.com>
Date:	Tue, 19 Oct 2010 16:34:32 +0900
From:	Minchan Kim <minchan.kim@...il.com>
To:	Shaohua Li <shaohua.li@...el.com>
Cc:	"Wu, Fengguang" <fengguang.wu@...el.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Neil Brown <neilb@...e.de>, Rik van Riel <riel@...hat.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>
Subject: Re: Deadlock possibly caused by too_many_isolated.

On Tue, Oct 19, 2010 at 4:15 PM, Shaohua Li <shaohua.li@...el.com> wrote:
> On Tue, Oct 19, 2010 at 11:21:45AM +0800, Shaohua Li wrote:
>> On Tue, Oct 19, 2010 at 11:09:29AM +0800, Minchan Kim wrote:
>> > On Tue, Oct 19, 2010 at 12:05 PM, Wu Fengguang <fengguang.wu@...el.com> wrote:
>> > > On Tue, Oct 19, 2010 at 10:52:47AM +0800, Minchan Kim wrote:
>> > >> Hi Wu,
>> > >>
>> > >> On Tue, Oct 19, 2010 at 11:35 AM, Wu Fengguang <fengguang.wu@...el.com> wrote:
>> > >> >> @@ -2054,10 +2069,11 @@ rebalance:
>> > >> >>                 goto got_pg;
>> > >> >>
>> > >> >>         /*
>> > >> >> -        * If we failed to make any progress reclaiming, then we are
>> > >> >> -        * running out of options and have to consider going OOM
>> > >> >> +        * If we failed to make any progress reclaiming and there aren't
>> > >> >> +        * many parallel reclaiming, then we are unning out of options and
>> > >> >> +        * have to consider going OOM
>> > >> >>          */
>> > >> >> -       if (!did_some_progress) {
>> > >> >> +       if (!did_some_progress && !too_many_isolated_zone(preferred_zone)) {
>> > >> >>                 if ((gfp_mask & __GFP_FS) && !(gfp_mask & __GFP_NORETRY)) {
>> > >> >>                         if (oom_killer_disabled)
>> > >> >>                                 goto nopage;
>> > >> >
>> > >> > This is simply wrong.
>> > >> >
>> > >> > It disabled this block for 99% system because there won't be enough
>> > >> > tasks to make (!too_many_isolated_zone == true). As a result the LRU
>> > >> > will be scanned like mad and no task get OOMed when it should be.
>> > >>
>> > >> If !too_many_isolated_zone is false, it means there are already many
>> > >> direct reclaiming tasks.
>> > >> So they could exit reclaim path and !too_many_isolated_zone will be true.
>> > >> What am I missing now?
>> > >
>> > > Ah sorry, my brain get short circuited.. but I still feel uneasy with
>> > > this change. It's not fixing the root cause and won't prevent too many
>> > > LRU pages be isolated. It's too late to test too_many_isolated_zone()
>> > > after direct reclaim returns (after sleeping for a long time).
>> > >
>> >
>> > Intend to agree.
>> > I think root cause is a infinite looping in too_many_isolated holding FS lock.
>> > Would it be simple that too_many_isolated would be bail out after some try?
>> I'm wondering if we need too_many_isolated_zone logic. The do_try_to_free_pages
>> will return progress till all zones are unreclaimable. Assume before this we
>> don't oomkiller. If the direct reclaim fails but has progress, it will sleep.
> Not sure if this is clear. What I mean is we can delete too_many_isolated_zone,
> do_try_to_free_pages can still return 1 till all zones are unreclaimable. Before
> this direct reclaim will not oom, because it sees progress and will call congestion_wait
> to sleep. Am I missing anything?
>

You mean could we remove too_many_isolated? not too_many_isolated_zone. Right?
It it is, we can't.

Your saying is right.
do_try_to_free_pages can return 1 until all zones are unreclaimable.
But If we remove throttling logic which is too_many_isolated, too many
process can enter direct reclaim path and they can increase
zone->pages_scanned while zone_reclaimable pages are decreased by
isolation. At last, we could reach all zones are unreclaimable much
faster.




-- 
Kind regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/