linux-kernel - Re: [RFC PATCH 0/2] mm: multi-gen LRU: working set extensions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAJj2-QHxcC3u49Boh7t0Z2tZfhrg_W5uLqBcMDYmukJra8D22Q@mail.gmail.com>
Date:   Wed, 11 Jan 2023 17:38:37 -0800
From:   Yuanchu Xie <yuanchu@...gle.com>
To:     Michal Koutný <mkoutny@...e.com>
Cc:     Johannes Weiner <hannes@...xchg.org>,
        Michal Hocko <mhocko@...nel.org>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Yu Zhao <yuzhao@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Muchun Song <songmuchun@...edance.com>,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        cgroups@...r.kernel.org
Subject: Re: [RFC PATCH 0/2] mm: multi-gen LRU: working set extensions

On Wed, Jan 11, 2023 at 6:17 AM Michal Koutný <mkoutny@...e.com> wrote:
>
> On Wed, Dec 14, 2022 at 02:51:21PM -0800, Yuanchu Xie <yuanchu@...gle.com> wrote:
> > that's frequently used. The only missing pieces between MGLRU
> > generations and working set estimation are a consistent aging cadence
> > and an interface; we introduce the two additions.
> >
> > Periodic aging
> > ======
> > MGLRU Aging is currently driven by reclaim, so the amount of time
> > between generations is non-deterministic. With memcgs being aged
> > regularly, MGLRU generations become time-based working set information.
>
> Is this periodic aging specific to memcgs? IOW, periodic aging isn't
> needed without memcgs (~with root only)
> (Perhaps similar question to Aneeh's.)
Originally, I didn't see much value in periodic aging without memcgs,
as the main goal was to provide working set information.
Periodic aging might lead to MGLRU making better reclaim decisions,
but I don't have any benchmarks to back it up right now.

>
> > Use case: proactive reclaimer
> > ======
> > The proactive reclaimer sets the aging interval, and periodically reads
> > the page idle age stats, forming a working set estimation, which it then
> > calculates an amount to write to memory.reclaim.
> >
> > With the page idle age stats, a proactive reclaimer could calculate a
> > precise amount of memory to reclaim without continuously probing and
> > inducing reclaim.
>
> Could the aging be also made per-memcg? (Similar to memory.reclaim,
> possibly without the new kthread (if global reclaim's aging is enough).)
It is possible. We can have hierarchical aging, invoked by writing to
memory.aging with a time duration. For every child memcg, if its young
generation is older than (current time - specified duration), do
aging.
However, now we need a userspace tool to drive the aging, invoking
this interface every few seconds, since every memcg is aged at a
different cadence.
Having a kthread perform aging has the benefit of simplicity, gives a
source of truth for the aging interval, and makes the feature more
accessible. The application developers, if they want to take a look at
the page idle age stats, could do so without needing additional
ceremony.

Thanks,
Yuanchu