linux-kernel - Re: [RFC 0/4] Introduce unbalance proactive reclaim

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <8d28e404-d5cc-4098-b745-b59608b1a66f@vivo.com>
Date:   Wed, 8 Nov 2023 17:05:53 +0800
From:   Huan Yang <link@...o.com>
To:     Yosry Ahmed <yosryahmed@...gle.com>
Cc:     "Huang, Ying" <ying.huang@...el.com>, Tejun Heo <tj@...nel.org>,
        Zefan Li <lizefan.x@...edance.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Jonathan Corbet <corbet@....net>,
        Michal Hocko <mhocko@...nel.org>,
        Roman Gushchin <roman.gushchin@...ux.dev>,
        Shakeel Butt <shakeelb@...gle.com>,
        Muchun Song <muchun.song@...ux.dev>,
        Andrew Morton <akpm@...ux-foundation.org>,
        David Hildenbrand <david@...hat.com>,
        Matthew Wilcox <willy@...radead.org>,
        Kefeng Wang <wangkefeng.wang@...wei.com>,
        Peter Xu <peterx@...hat.com>,
        "Vishal Moola (Oracle)" <vishal.moola@...il.com>,
        Liu Shixin <liushixin2@...wei.com>,
        Hugh Dickins <hughd@...gle.com>, cgroups@...r.kernel.org,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org, opensource.kernel@...o.com
Subject: Re: [RFC 0/4] Introduce unbalance proactive reclaim


在 2023/11/8 17:00, Yosry Ahmed 写道:
> On Wed, Nov 8, 2023 at 12:22 AM Huan Yang <link@...o.com> wrote:
>>
>> 在 2023/11/8 16:14, Yosry Ahmed 写道:
>>> On Wed, Nov 8, 2023 at 12:11 AM Huang, Ying <ying.huang@...el.com> wrote:
>>>> Huan Yang <link@...o.com> writes:
>>>>
>>>>> HI Huang, Ying
>>>>>
>>>>> Thanks for reply.
>>>>>
>>>>> 在 2023/11/8 15:35, Huang, Ying 写道:
>>>>>> Huan Yang <link@...o.com> writes:
>>>>>>
>>>>>>> In some cases, we need to selectively reclaim file pages or anonymous
>>>>>>> pages in an unbalanced manner.
>>>>>>>
>>>>>>> For example, when an application is pushed to the background and frozen,
>>>>>>> it may not be opened for a long time, and we can safely reclaim the
>>>>>>> application's anonymous pages, but we do not want to touch the file pages.
>>>>>>>
>>>>>>> This patchset extends the proactive reclaim interface to achieve
>>>>>>> unbalanced reclamation. Users can control the reclamation tendency by
>>>>>>> inputting swappiness under the original interface. Specifically, users
>>>>>>> can input special values to extremely reclaim specific pages.
>>>>>>    From mem_cgroup_swappiness(), cgroupv2 doesn't have per-cgroup
>>>>>> swappiness.  So you need to add that firstly?
>>>>> Sorry for this mistake, we always work on cgroupv1, so, not notice
>>>>> this commit 4550c4e, thank your for point that.
>>>>>
>>>>> I see this commit comment that `that's a different discussion`, but,
>>>>> to implements this, I will try add.
>>>>>
>>>>>>> Example:
>>>>>>>       echo "1G" 200 > memory.reclaim (only reclaim anon)
>>>>>>>         echo "1G" 0  > memory.reclaim (only reclaim file)
>>>>>>>         echo "1G" 1  > memory.reclaim (only reclaim file)
>>>>>>>
>>>>>>> Note that when performing unbalanced reclamation, the cgroup swappiness
>>>>>>> will be temporarily adjusted dynamically to the input value. Therefore,
>>>>>>> if the cgroup swappiness is further modified during runtime, there may
>>>>>>> be some errors.
>>>>>> If cgroup swappiness will be adjusted temporarily, why not just change
>>>>>> it via a script before/after proactive reclaiming?
>>>>> IMO, this unbalance reclaim only takes effect for a single command,
>>>>> so if it is pre-set using a script, the judgment of the reclamation tendency
>>>>> may become complicated.
>>>> If swappiness == 0, then we will only reclaim file pages.  If swappiness
>>>> == 200, then we may still reclaim file pages.  So you need a way to
>>>> reclaim only anon pages?
>>>>
>>>> If so, can we use some special swappiness value to specify that?  I
>>>> don't know whether use 200 will cause regression.  If so, we may need
>>>> some other value, e.g. >= 65536.
>>> I don't think swappiness is the answer here. This has been discussed a
>>> while back, please see my response. As you mentioned, swappiness may
>>> be ignored by the kernel in some cases, and its behavior has
>>> historically changed before.
>> For type base, reclaim can have direct tendencies as well. It's good.
>> But, what if
>> we only want to make small adjustments to the reclamation ratio?
>> Of course, sometimes swappiness may become ineffective.
>>
> Is there a real use case for this? I think it's difficult to reason
> about swappiness and make small adjustments to the file/anon ratio
> based on it. I'd prefer a more concrete implementation.

For example, swappiness=170 to try hard reclaim anon, a little pressure to
reclaim file(expect reclaim clean file). In theory, this method can help 
reduce
memory pressure.
Or else, reclaim 80% anon and trim 5% code file control is good when it is
detected that an application has been frozen for a period of time.