linux-kernel - Re: [PATCH v1 09/14] mm: multigenerational lru: mm

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YFAsjP7NIZM5Ld+m@google.com>
Date:   Mon, 15 Mar 2021 21:57:00 -0600
From:   Yu Zhao <yuzhao@...gle.com>
To:     "Huang, Ying" <ying.huang@...el.com>
Cc:     Rik van Riel <riel@...riel.com>, linux-mm@...ck.org,
        Alex Shi <alex.shi@...ux.alibaba.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Hillf Danton <hdanton@...a.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        Matthew Wilcox <willy@...radead.org>,
        Mel Gorman <mgorman@...e.de>, Michal Hocko <mhocko@...e.com>,
        Roman Gushchin <guro@...com>, Vlastimil Babka <vbabka@...e.cz>,
        Wei Yang <richard.weiyang@...ux.alibaba.com>,
        Yang Shi <shy828301@...il.com>, linux-kernel@...r.kernel.org,
        page-reclaim@...gle.com
Subject: Re: [PATCH v1 09/14] mm: multigenerational lru: mm_struct list

On Tue, Mar 16, 2021 at 10:07:36AM +0800, Huang, Ying wrote:
> Rik van Riel <riel@...riel.com> writes:
> 
> > On Sat, 2021-03-13 at 00:57 -0700, Yu Zhao wrote:
> >
> >> +/*
> >> + * After pages are faulted in, they become the youngest generation.
> >> They must
> >> + * go through aging process twice before they can be evicted. After
> >> first scan,
> >> + * their accessed bit set during initial faults are cleared and they
> >> become the
> >> + * second youngest generation. And second scan makes sure they
> >> haven't been used
> >> + * since the first.
> >> + */
> >
> > I have to wonder if the reductions in OOM kills and 
> > low-memory tab discards is due to this aging policy
> > change, rather than from the switch to virtual scanning.

There are no policy changes per se. The current page reclaim also
scans a faulted-in page at least twice before it can reclaim it.
That said, the new aging yields a better overall result because it
discovers every page that has been referenced since the last scan,
in addition to what Ying has mentioned. The current page scan stops
stops once it finds enough candidates, which may seem more
efficiently, but actually pays the price for not finding the best.

> If my understanding were correct, the temperature of the processes is
> considered in addition to that of the individual pages.  That is, the
> pages of the processes that haven't been scheduled after the previous
> scanning will not be scanned.  I guess that this helps OOM kills?

Yes, that's correct.

> If so, how about just take advantage of that information for OOM killing
> and page reclaiming?  For example, if a process hasn't been scheduled
> for long time, just reclaim its private pages.

This is how it works. Pages that haven't been scanned grow older
automatically because those that have been scanned will be tagged with
younger generation numbers. Eviction does bucket sort based on
generation numbers and attacks the oldest.