linux-kernel - Re: [RFC PATCH 12/26] mm: page_alloc: per-migratetype free counts

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230421163234.GF320347@cmpxchg.org>
Date:   Fri, 21 Apr 2023 12:32:34 -0400
From:   Johannes Weiner <hannes@...xchg.org>
To:     Mel Gorman <mgorman@...hsingularity.net>
Cc:     linux-mm@...ck.org, Kaiyang Zhao <kaiyang2@...cmu.edu>,
        Vlastimil Babka <vbabka@...e.cz>,
        David Rientjes <rientjes@...gle.com>,
        linux-kernel@...r.kernel.org, kernel-team@...com
Subject: Re: [RFC PATCH 12/26] mm: page_alloc: per-migratetype free counts

On Fri, Apr 21, 2023 at 05:03:20PM +0100, Mel Gorman wrote:
> On Fri, Apr 21, 2023 at 11:35:01AM -0400, Johannes Weiner wrote:
> > On Fri, Apr 21, 2023 at 03:28:41PM +0100, Mel Gorman wrote:
> > > On Tue, Apr 18, 2023 at 03:12:59PM -0400, Johannes Weiner wrote:
> > > > Increase visibility into the defragmentation behavior by tracking and
> > > > reporting per-migratetype free counters.
> > > > 
> > > > Subsequent patches will also use those counters to make more targeted
> > > > reclaim/compaction decisions.
> > > > 
> > > > Signed-off-by: Johannes Weiner <hannes@...xchg.org>
> > > 
> > > Visibility into fragmentation behaviour is information that is
> > > almost certainly only useful to a developer and even then, there is
> > > /proc/pagetypeinfo. At minimum, move this patch to later in the series
> > > but I'm skeptical about its benefit.
> > 
> > Having them available in the memory dump (OOM, sysrq) was essential
> > while debugging problems in later patches. For OOMs or lockups,
> > pagetypeinfo isn't available. It would be useful to have them included
> > in user reports if any issues pop up.
> > 
> 
> OOM+sysrq could optionally take the very expensive step of traversing the
> lists to get the count so yes, it helps debugging, but not necessarily
> critical.
> 
> > They're used internally in several places later on, too.
> > 
> 
> I did see that for deciding the suitability for compaction. Minimally, put
> the patches adjacent in the series and later if possible so that the series
> can be taken in parts. There are a lot of patches that should be relatively
> uncontroversial so maybe make "mm: page_alloc: introduce MIGRATE_FREE" the
> pivot point between incremental improvements and "everything on and after
> this patch is relatively high risk, could excessively compact/reclaim,
> could livelock etc".

Okay, I see now where you're coming from. That's good feedback.

Actually most of the patches work toward the final goal of managing
free memory in whole blocks. The only exception are the block pages,
the nofs deadlock, the page_isolation kernel doc, and *maybe* the
should_[compact|reclaim]_retry cleanups. I tried to find the
standalone value in each of the prep patches as well to avoid
forward-referencing in the series too much. But obviously these
standalone reasons tend to be on the weak side.

I'll rework the changelogs (and patch ordering) where applicable to
try to make the dependencies clearer.