linux-kernel - Re: [PATCH 2/3] check unevictable flag in lumy reclaim v2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <d3a71da3a374d278e5fb0b1f2cdff71e.squirrel@webmail-b.css.fujitsu.com>
Date:	Thu, 11 Jun 2009 21:18:22 +0900 (JST)
From:	"KAMEZAWA Hiroyuki" <kamezawa.hiroyu@...fujitsu.com>
To:	"Minchan Kim" <minchan.kim@...il.com>
Cc:	"KAMEZAWA Hiroyuki" <kamezawa.hiroyu@...fujitsu.com>,
	"KOSAKI Motohiro" <kosaki.motohiro@...fujitsu.com>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"nishimura@....nes.nec.co.jp" <nishimura@....nes.nec.co.jp>,
	"balbir@...ux.vnet.ibm.com" <balbir@...ux.vnet.ibm.com>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	apw@...onical.com, riel@...hat.com, mel@....ul.ie,
	"Lee Schermerhorn" <lee.schermerhorn@...com>
Subject: Re: [PATCH 2/3] check unevictable flag in lumy reclaim v2

Minchan Kim wrote:
> 2009/6/11 KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>:
>> Minchan Kim さん wrote:
>>> On Thu, Jun 11, 2009 at 5:38 PM, KAMEZAWA
>>> Hiroyuki<kamezawa.hiroyu@...fujitsu.com> wrote:
>>>> How about this ?
>>>>
>>>> From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
>>>>
>>>> Lumpy reclaim check pages from their pfn. Then, it can find
>>>> unevictable
>>>> pages
>>>> in its loop.
>>>> Abort lumpy reclaim when we find Unevictable page, we never get a lump
>>>> of pages for requested order.
>>>>
>>>> Changelog: v1->v2
>>>> ?- rewrote commet.
>>>>
>>>> Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
>>>> ---
>>>> ?mm/vmscan.c | ? ?9 +++++++++
>>>> ?1 file changed, 9 insertions(+)
>>>>
>>>> Index: lumpy-reclaim-trial/mm/vmscan.c
>>>> ===================================================================
>>>> --- lumpy-reclaim-trial.orig/mm/vmscan.c
>>>> +++ lumpy-reclaim-trial/mm/vmscan.c
>>>> @@ -936,6 +936,15 @@ static unsigned long isolate_lru_pages(u
>>>> ? ? ? ? ? ? ? ? ? ? ? ?/* Check that we have not crossed a zone
>>>> boundary. */
>>>> ? ? ? ? ? ? ? ? ? ? ? ?if (unlikely(page_zone_id(cursor_page) !=
>>>> zone_id))
>>>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?continue;
>>>> + ? ? ? ? ? ? ? ? ? ? ? /*
>>>> + ? ? ? ? ? ? ? ? ? ? ? ?* We tries to free all pages in this range to
>>>> create
>>>> + ? ? ? ? ? ? ? ? ? ? ? ?* a free large page. Then, if the range
>>>> includes a page
>>>> + ? ? ? ? ? ? ? ? ? ? ? ?* never be reclaimed, we have no reason to do
>>>> more.
>>>> + ? ? ? ? ? ? ? ? ? ? ? ?* PageUnevictable page is not a page which
>>>> can
>>>> be
>>>> + ? ? ? ? ? ? ? ? ? ? ? ?* easily freed. Abort this scan now.
>>>> + ? ? ? ? ? ? ? ? ? ? ? ?*/
>>>> + ? ? ? ? ? ? ? ? ? ? ? if (unlikely(PageUnevictable(cursor_page)))
>>>> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? break;
>>>
>>> __isolate_lru_pages already checked PageUnevictable to return error.
>>> I want to remove repeated check although it is trivial.
>>>
>>> By your patch, It seems to remove PageUnevictable check in
>>> __isolate_lru_pages.
>>>
>> yes.
>>
>>> But I know that. If we remove PageUnevictable check in
>>> __isolate_lru_pages, it can't go into BUG in non-lumpy case. ( I
>>> mentioned following as code)
>>>
>> In non-lumpy case, we'll never see Unevictable, maybe.
>
> I think so if it doesn't happen RAM failure.
> AFAIK, Unevictable check didn't related with RAM failure.
>
>>
>>> ? ? ? ? ? ? ? ? case -EBUSY:
>>> ? ? ? ? ? ? ? ? ? ? ? ? /* else it is being freed elsewhere */
>>> ? ? ? ? ? ? ? ? ? ? ? ? list_move(&page->lru, src);
>>> ? ? ? ? ? ? ? ? ? ? ? ? continue;
>>>
>>> ? ? ? ? ? ? ? ? default:
>>> ? ? ? ? ? ? ? ? ? ? ? ? BUG();
>>> ? ? ? ? ? ? ? ? }
>>>
>>>
>>> It means we can remove BUG in non-lumpy case and then add BUG into
>>> __isolate_lru_pages directly.
>>>
>>> If we can do it, we can remove unnecessary PageUnevictable check in
>>> __isolate_lru_page.
>>>
>> Hmm, but Unevicable check had tons of troubles at its implementation
>> and I don't want to do it at once.
>
> I think it's not a big problem.
> As comment said, the check's goal is to prevent in lumpy case.
>         /*
>          * When this function is being called for lumpy reclaim, we
>          * initially look into all LRU pages, active, inactive and
>          * unevictable; only give shrink_page_list evictable pages.
>          */
>         if (PageUnevictable(page))
>                 return ret;
>
> So I think we can remove this check.
>
agreed.

>>> I am not sure this is right in case of memcg.
>>>
>> I think we don't see Unevictable in memcg's path if my memcg-lru code
>> works as designed.
>>
>> I'll postpone this patch for a while until my brain works well.
>
> If you have a concern about that, how about this ?
> (This code will be hunk since gmail webserver always mangle. Pz,forgive
> me)
> Also, we can CC original authors.
>
I'll schedule this optimization/clean up for unevictable case in queue.
Thank you for inputs.

But it's now merge-window, I'd like to push bugfix first.(1/3 and 3/3)
I'd like to scheule Unevictable case fix after rc1(when mmotm stack seems
to be pushed out to Linus.)
And I'll add
int __isolate_lru_page(...)
{
     VM_BUG_ON(PageUnevictable(page));
}
as sanity check for mmotm test time.

Thank you for all your help.
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/