linux-kernel - Re: [External] Re: [RFC PATCH 0/2] Add disable_unmap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZtBglyqZz_uGDnOS@tiehlicka>
Date: Thu, 29 Aug 2024 13:50:47 +0200
From: Michal Hocko <mhocko@...e.com>
To: Zhongkun He <hezhongkun.hzk@...edance.com>
Cc: akpm@...ux-foundation.org, hannes@...xchg.org, roman.gushchin@...ux.dev,
	shakeel.butt@...ux.dev, muchun.song@...ux.dev,
	lizefan.x@...edance.com, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org, cgroups@...r.kernel.org
Subject: Re: [External] Re: [RFC PATCH 0/2] Add disable_unmap_file arg to
 memory.reclaim

On Thu 29-08-24 18:37:07, Zhongkun He wrote:
> On Thu, Aug 29, 2024 at 6:24 PM Michal Hocko <mhocko@...e.com> wrote:
> >
> > On Thu 29-08-24 18:19:16, Zhongkun He wrote:
> > > This patch proposes augmenting the memory.reclaim interface with a
> > > disable_unmap_file argument that will skip the mapped pages in
> > > that reclaim attempt.
> > >
> > > For example:
> > >
> > > echo "2M disable_unmap_file" > /sys/fs/cgroup/test/memory.reclaim
> > >
> > > will perform reclaim on the test cgroup with no mapped file page.
> > >
> > > The memory.reclaim is a useful interface. We can carry out proactive
> > > memory reclaim in the user space, which can increase the utilization
> > > rate of memory.
> > >
> > > In the actual usage scenarios, we found that when there are sufficient
> > > anonymous pages, mapped file pages with a relatively small proportion
> > > would still be reclaimed. This is likely to cause an increase in
> > > refaults and an increase in task delay, because mapped file pages
> > > usually include important executable codes, data, and shared libraries,
> > > etc. According to the verified situation, if we can skip this part of
> > > the memory, the task delay will be reduced.
> >
> > Do you have examples of workloads where this is demonstrably helps and
> > cannot be tuned via swappiness?
> 
> Sorry, I put the test workload in the second patch. Please have a look.

I have missed those as they are not threaded to the cover letter. You
can either use --in-reply-to when sending patches separately from the
cover letter or use can use --compose/--cover-leter when sending patches
through git-send-email

> Even if there are sufficient anonymous pages and a small number of
> page cache and mapped file pages, mapped file pages will still be reclaimed.
> Here is an example of anonymous pages being sufficient but mapped
> file pages still being reclaimed:
> Swappiness has been set to the maximum value.
> 
> cat memory.stat | grep -wE 'anon|file|file_mapped'
> anon 3406462976
> file 332967936
> file_mapped 300302336
> 
> echo 1g > memory.reclaim swappiness=200 > memory.reclaim
> cat memory.stat | grep -wE 'anon|file|file_mapped'
> anon 2613276672
> file 52523008
> file_mapped 30982144

This seems to be 73% (ano) vs 27% (file) balance. 90% of the 
file LRU seems to be mapped which matches 90% of file LRU reclaimed
memory to be mapped. So the reclaim is proportional there.

But I do understand that this is still unexpected when swappiness=200
should make reclaim anon oriented. Is this MGLRU or regular LRU
implementation?

Is this some artificial workload or something real world?


-- 
Michal Hocko
SUSE Labs