linux-kernel - Re: [PATCH v12] mm: report per-page metadata information

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1e1b89af-3b7a-7400-cfd7-d22a101955de@google.com>
Date: Fri, 24 May 2024 13:00:06 -0700 (PDT)
From: David Rientjes <rientjes@...gle.com>
To: Sourav Panda <souravpanda@...gle.com>
cc: corbet@....net, gregkh@...uxfoundation.org, rafael@...nel.org, 
    Andrew Morton <akpm@...ux-foundation.org>, mike.kravetz@...cle.com, 
    muchun.song@...ux.dev, rppt@...nel.org, david@...hat.com, 
    rdunlap@...radead.org, chenlinxuan@...ontech.com, yang.yang29@....com.cn, 
    tomas.mudrunka@...il.com, bhelgaas@...gle.com, ivan@...udflare.com, 
    pasha.tatashin@...een.com, yosryahmed@...gle.com, hannes@...xchg.org, 
    shakeelb@...gle.com, kirill.shutemov@...ux.intel.com, 
    wangkefeng.wang@...wei.com, adobriyan@...il.com, 
    Vlastimil Babka <vbabka@...e.cz>, 
    "Liam R. Howlett" <Liam.Howlett@...cle.com>, surenb@...gle.com, 
    linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org, 
    linux-doc@...r.kernel.org, linux-mm@...ck.org, 
    Matthew Wilcox <willy@...radead.org>, weixugc@...gle.com
Subject: Re: [PATCH v12] mm: report per-page metadata information

On Sat, 18 May 2024, David Rientjes wrote:

> On Sun, 12 May 2024, Sourav Panda wrote:
> 
> > Today, we do not have any observability of per-page metadata
> > and how much it takes away from the machine capacity. Thus,
> > we want to describe the amount of memory that is going towards
> > per-page metadata, which can vary depending on build
> > configuration, machine architecture, and system use.
> > 
> > This patch adds 2 fields to /proc/vmstat that can used as shown
> > below:
> > 
> > Accounting per-page metadata allocated by boot-allocator:
> > 	/proc/vmstat:nr_memmap_boot * PAGE_SIZE
> > 
> > Accounting per-page metadata allocated by buddy-allocator:
> > 	/proc/vmstat:nr_memmap * PAGE_SIZE
> > 
> > Accounting total Perpage metadata allocated on the machine:
> > 	(/proc/vmstat:nr_memmap_boot +
> > 	 /proc/vmstat:nr_memmap) * PAGE_SIZE
> > 
> > Utility for userspace:
> > 
> > Observability: Describe the amount of memory overhead that is
> > going to per-page metadata on the system at any given time since
> > this overhead is not currently observable.
> > 
> > Debugging: Tracking the changes or absolute value in struct pages
> > can help detect anomalies as they can be correlated with other
> > metrics in the machine (e.g., memtotal, number of huge pages,
> > etc).
> > 
> > page_ext overheads: Some kernel features such as page_owner
> > page_table_check that use page_ext can be optionally enabled via
> > kernel parameters. Having the total per-page metadata information
> > helps users precisely measure impact. Furthermore, page-metadata
> > metrics will reflect the amount of struct pages reliquished
> > (or overhead reduced) when hugetlbfs pages are reserved which
> > will vary depending on whether hugetlb vmemmap optimization is
> > enabled or not.
> > 
> > For background and results see:
> > lore.kernel.org/all/20240220214558.3377482-1-souravpanda@...gle.com
> > 
> > Signed-off-by: Sourav Panda <souravpanda@...gle.com>
> > Reviewed-by: Pasha Tatashin <pasha.tatashin@...een.com>
> 
> Acked-by: David Rientjes <rientjes@...gle.com>
> 

This would be a very useful extension to be able to provide observability 
of per-page metadata overhead and the impact of things like HVO on the 
overall footprint.  Today, we don't have observability for this memory 
overhead.

Andrew, anything else that can be addressed before this is eligible for 
staging in MM unstable?