linux-kernel - Re: [RFC 0/4] Introduce unbalance proactive reclaim

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZUzUhWsrzDQwMKQ-@tiehlicka>
Date:   Thu, 9 Nov 2023 13:45:57 +0100
From:   Michal Hocko <mhocko@...e.com>
To:     Huan Yang <link@...o.com>
Cc:     Tejun Heo <tj@...nel.org>, Zefan Li <lizefan.x@...edance.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Jonathan Corbet <corbet@....net>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Shakeel Butt <shakeelb@...gle.com>,
        Muchun Song <muchun.song@...ux.dev>,
        Andrew Morton <akpm@...ux-foundation.org>,
        David Hildenbrand <david@...hat.com>,
        Matthew Wilcox <willy@...radead.org>,
        Huang Ying <ying.huang@...el.com>,
        Kefeng Wang <wangkefeng.wang@...wei.com>,
        Peter Xu <peterx@...hat.com>,
        "Vishal Moola (Oracle)" <vishal.moola@...il.com>,
        Yosry Ahmed <yosryahmed@...gle.com>,
        Liu Shixin <liushixin2@...wei.com>,
        Hugh Dickins <hughd@...gle.com>, cgroups@...r.kernel.org,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org, opensource.kernel@...o.com
Subject: Re: [RFC 0/4] Introduce unbalance proactive reclaim

On Thu 09-11-23 18:55:09, Huan Yang wrote:
> 
> 在 2023/11/9 17:53, Michal Hocko 写道:
> > [Some people who received this message don't often get email from mhocko@...e.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
> > 
> > On Thu 09-11-23 09:56:46, Huan Yang wrote:
> > > 在 2023/11/8 22:06, Michal Hocko 写道:
> > > > [Some people who received this message don't often get email from mhocko@...e.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
> > > > 
> > > > On Wed 08-11-23 14:58:11, Huan Yang wrote:
> > > > > In some cases, we need to selectively reclaim file pages or anonymous
> > > > > pages in an unbalanced manner.
> > > > > 
> > > > > For example, when an application is pushed to the background and frozen,
> > > > > it may not be opened for a long time, and we can safely reclaim the
> > > > > application's anonymous pages, but we do not want to touch the file pages.
> > > > Could you explain why? And also why do you need to swap out in that
> > > > case?
> > > When an application is frozen, it usually means that we predict that
> > > it will not be used for a long time. In order to proactively save some
> > > memory, our strategy will choose to compress the application's private
> > > data into zram. And we will also select some of the cold application
> > > data that we think is in zram and swap it out.
> > > 
> > > The above operations assume that anonymous pages are private to the
> > > application.  After the application is frozen, compressing these pages
> > > into zram can save memory to some extent without worrying about
> > > frequent refaults.
> > Why don't you rely on the default reclaim heuristics? In other words do
> As I mentioned earlier, the madvise approach may not be suitable for my
> needs.

I was asking about default reclaim behavior not madvise here.

> > you have any numbers showing that a selective reclaim results in a much
> 
> In the mobile field, we have a core metric called application residency.

As already pointed out in other reply, make sure you explain this so
that we, who are not active in mobile field, can understand the metric,
how it is affected by the tooling relying on this interface.

> This mechanism can help us improve the application residency if we can
> provide a good freeze detection and proactive reclamation policy.
> 
> I can only provide specific data from our internal tests, and it may
> be older data, and it tested using cgroup v1:
> 
> In 12G ram phone, app residency improve from 29 to 38.

cgroup v1 is in maintenance mode and new extension would need to pass
even a higher feasibility test than v2 based interface. Also make sure
that you are testing the current upstream kernel.

Also let me stress out that you are proposing an extension to the user
visible API and we will have to maintain that for ever. So make sure
your justification is solid and understandable. 
-- 
Michal Hocko
SUSE Labs