linux-kernel - Re: [PATCH v2] mm/madvise: add vmstat statistics for madvise

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Y9MWC8xh8gOKDrUU@google.com>
Date:   Thu, 26 Jan 2023 16:08:43 -0800
From:   Minchan Kim <minchan@...nel.org>
To:     Michal Hocko <mhocko@...e.com>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Suren Baghdasaryan <surenb@...gle.com>,
        Matthew Wilcox <willy@...radead.org>,
        linux-mm <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v2] mm/madvise: add vmstat statistics for
 madvise_[cold|pageout]

On Thu, Jan 26, 2023 at 08:58:57PM +0100, Michal Hocko wrote:
> On Thu 26-01-23 09:10:46, Minchan Kim wrote:
> > On Thu, Jan 26, 2023 at 09:50:37AM +0100, Michal Hocko wrote:
> [...]
> > > I suspect you try to mimic pgscan/pgsteal effectivness metric on the
> > > address space but that is a fundamentally different thing.
> > 
> > I don't see anything different, fundamentally.
> 
> OK, this really explains our disconnect here. Your metric reports
> nr_page_tables (nr_scanned) and number of aged and potentially reclaimed
> pages. You do not know whether that reclaim was successful. So you
> effectively learn how many pages have already been unmapped before your
> call. Can this be sometimes useful? Probably yes. Does it say anything
> about the reclaim efficiency? I do not think so. You could have hit
> pinned pages or countless other conditions why those pages couldn't have
> been reclaimed and they have stayed mapped after madvise call.
> 
> pgsteal tells you how many pages from those scanned have been reclaimed.
> See the difference?

That's why my previous version kept counting exact number of reclaimed/
deactivated pages but I changed mind since I observed majority of failure
happened from already-paged-out ranges and shared pages rather than minor
countless other conditions in real practice. Without finding present pages,
the mavise hints couldn't do anything from the beginning and that's the
major cost we are facing.

Saing again, I don't think the global stat could cover all the minor
you are insisting and I agree tracepoint could do better jobs to pinpoint
root causes but the global stat still have a role to provides basic ground
to sense abnormal and guides us moving next steps with easier interface/
efficient way.

> 
> Also I do not find information about how many non-present ptes have
> been scann super interesting. Sure that is a burnt time as well but to
> me it would be much more valuable information to see how many of those
> resident could have been actually reclaimed. Because that tells whether
> your reclaim target was a good choice and IMHO that is a valuable
> information for user space memory reclaim agent.

That's exactly what I had in previous version. If you believe it's right
direction, I am okay.

> 
> Again consider a large sparsely mapped memory but mostly inactive memory
> and a condensed active one with the same rss. The reclaim could have
> been successful for the former while not on the latter. Your matric
> would give a rather misleading numbers, don't you think?
> -- 
> Michal Hocko
> SUSE Labs