linux-kernel - Re: [PATCH v3] mm: add thp_utilization metrics to /proc/thp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 5 Aug 2022 20:24:25 +0100
From:   Matthew Wilcox <willy@...radead.org>
To:     Rik van Riel <riel@...com>
Cc:     "Alex Zhu (Kernel)" <alexlzhu@...com>,
        Kernel Team <Kernel-team@...com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>
Subject: Re: [PATCH v3] mm: add thp_utilization metrics to
 /proc/thp_utilization

On Fri, Aug 05, 2022 at 07:04:30PM +0000, Rik van Riel wrote:
> On Fri, 2022-08-05 at 19:50 +0100, Matthew Wilcox wrote:
> > > 
> > > This change introduces a tool that scans through all of physical
> > > memory for anonymous THPs and groups them into buckets based
> > > on utilization. It also includes an interface under
> > > /proc/thp_utilization.
> > 
> > OK, so I understand why we want to do the scanning, but why do we
> > want to
> > report it to userspace at all?  And if we do, why do we want to do it
> > in
> > this format?  AFAIK, there's nothing userspace can do with the
> > information
> > "93% of your THPs are underutilised".  If there was something
> > userspace
> > could do about it, wouldn't it need to know which ones?
> > 
> > Isn't the real solution here entirely in-kernel?  This scanning
> > thread
> > you've created should be the one splitting the THPs.  And anyway,
> > isn't
> > this exactly the kind of thing that DAMON was invented for?
> 
> Alex does have an (in kernel) shrinker that can reclaim
> underutilized THPs in order to free memory.

Ah!  So when that exists, this interface tells us "how well" we're doing.

> This is a regular shrinker called from kswapd. I am not
> sure a shrinker going through the DAMON infrastructure
> would be any smaller, faster, or better. OTOH, DAMON
> does allow for more flexible policy...
> 
> Getting some info on the effectiveness of the shrinker
> seems useful, though. Could debugfs be a better place?
> Should this be resubmitted together with the shrinker
> code that makes this more useful?

Yeah, debugfs seems like a better place.  And I'd love to see the shrinker
code.  Before you mentioned that I was having all kinds of peculiar
feelings about this code.  For example, suppose you have incredibly hot
256kB of data, but the other 1792kB of data are never touched ... that
could cause us to do entirely the wrong thing and break up this THP.
Having it as a shrinker makes sense because the hot 256kB will keep the
THP from reaching the end of the list and being reclaimed.