linux-kernel - Re: [PATCH] memcg: provide reclaim stats via 'memory.reclaim'

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YodDaFVeU33bu7yQ@dhcp22.suse.cz>
Date:   Fri, 20 May 2022 09:29:44 +0200
From:   Michal Hocko <mhocko@...e.com>
To:     Vaibhav Jain <vaibhav@...ux.ibm.com>
Cc:     cgroups@...r.kernel.org, linux-doc@...r.kernel.org,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        Tejun Heo <tj@...nel.org>, Zefan Li <lizefan.x@...edance.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Jonathan Corbet <corbet@....net>,
        Vladimir Davydov <vdavydov.dev@...il.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        "Aneesh Kumar K . V" <aneesh.kumar@...ux.ibm.com>,
        Shakeel Butt <shakeelb@...gle.com>,
        Yosry Ahmed <yosryahmed@...gle.com>
Subject: Re: [PATCH] memcg: provide reclaim stats via 'memory.reclaim'

On Fri 20-05-22 10:45:43, Vaibhav Jain wrote:
> 
> Thanks for looking into this patch Michal,
> 
> Michal Hocko <mhocko@...e.com> writes:
> 
> > On Thu 19-05-22 04:08:15, Vaibhav Jain wrote:
> >> [1] Provides a way for user-space to trigger proactive reclaim by introducing
> >> a write-only memcg file 'memory.reclaim'. However reclaim stats like number
> >> of pages scanned and reclaimed is still not directly available to the
> >> user-space.
> >> 
> >> This patch proposes to extend [1] to make the memcg file 'memory.reclaim'
> >> readable which returns the number of pages scanned / reclaimed during the
> >> reclaim process from 'struct vmpressure' associated with each memcg. This should
> >> let user-space asses how successful proactive reclaim triggered from memcg
> >> 'memory.reclaim' was ?
> >> 
> >> With the patch following command flow is expected:
> >> 
> >>  # echo "1M" > memory.reclaim
> >> 
> >>  # cat memory.reclaim
> >>    scanned 76
> >>    reclaimed 32
> >
> > Why cannot you use memory.stat? Sure it would require to iterate over
> > the reclaimed hierarchy but the information about scanned and reclaimed
> > pages as well as other potentially useful stats is there.
> 
> Agree that "memory.stat" is more suitable for scanned/reclaimed stats as
> it already is exposing bunch of other stats.
> 
> The discussion on this patch however seems to have split into two parts:
> 
> 1. Is it a good idea to expose nr_scanned/nr_reclaimed to users-space
> and if yes how ?
> 
> IMHO, I think it will be better to expose this info via 'memory.stat' as it
> can be useful insight into the reclaim efficiency  and vmpressure.

We already do that with some more metrics
pgrefill 9801926
pgscan 27329762
pgsteal 22715987
pgactivate 250691267
pgdeactivate 9521843
pglazyfree 0
pglazyfreed 0
 
> 2. Will it be useful to provide feedback to userspace when it writes to
> 'memory.reclaim' on how much memory has been reclaimed ?
> 
> IMHO, this will be a useful feeback to userspace to better adjust future
> proactive reclaim requests via 'memory.reclaim'

How precise this information should be? A very simplistic approach would
be
cp memory.stat stats.before
echo $WHATEVER > memory.reclaim
cp memory.stat stats.after

This will obviously contain also activity outside of the explicitly
triggered reclaim (racing background/direct reclaim) but isn't that what
actually matters? Are there any cases where the only metric you care
about is the triggered reclaim in isolation?

-- 
Michal Hocko
SUSE Labs