linux-kernel - Re: [PATCH v13] mm: report per-page metadata information

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+CK2bAfgamzFos1M-6AtozEDwRPJzARJOmccfZ=uzKyJ7w=kQ@mail.gmail.com>
Date: Mon, 5 Aug 2024 14:40:48 -0400
From: Pasha Tatashin <pasha.tatashin@...een.com>
To: Alison Schofield <alison.schofield@...el.com>
Cc: Sourav Panda <souravpanda@...gle.com>, corbet@....net, gregkh@...uxfoundation.org, 
	rafael@...nel.org, akpm@...ux-foundation.org, mike.kravetz@...cle.com, 
	muchun.song@...ux.dev, rppt@...nel.org, david@...hat.com, 
	rdunlap@...radead.org, chenlinxuan@...ontech.com, yang.yang29@....com.cn, 
	tomas.mudrunka@...il.com, bhelgaas@...gle.com, ivan@...udflare.com, 
	yosryahmed@...gle.com, hannes@...xchg.org, shakeelb@...gle.com, 
	kirill.shutemov@...ux.intel.com, wangkefeng.wang@...wei.com, 
	adobriyan@...il.com, vbabka@...e.cz, Liam.Howlett@...cle.com, 
	surenb@...gle.com, linux-kernel@...r.kernel.org, 
	linux-fsdevel@...r.kernel.org, linux-doc@...r.kernel.org, linux-mm@...ck.org, 
	willy@...radead.org, weixugc@...gle.com, David Rientjes <rientjes@...gle.com>, 
	nvdimm@...ts.linux.dev, linux-cxl@...r.kernel.org, yi.zhang@...hat.com
Subject: Re: [PATCH v13] mm: report per-page metadata information

On Fri, Aug 2, 2024 at 3:02 PM Alison Schofield
<alison.schofield@...el.com> wrote:
>
> ++ nvdimm, linux-cxl, Yu Zhang
>
> On Wed, Jun 05, 2024 at 10:27:51PM +0000, Sourav Panda wrote:
> > Today, we do not have any observability of per-page metadata
> > and how much it takes away from the machine capacity. Thus,
> > we want to describe the amount of memory that is going towards
> > per-page metadata, which can vary depending on build
> > configuration, machine architecture, and system use.
> >
> > This patch adds 2 fields to /proc/vmstat that can used as shown
> > below:
> >
> > Accounting per-page metadata allocated by boot-allocator:
> >       /proc/vmstat:nr_memmap_boot * PAGE_SIZE
> >
> > Accounting per-page metadata allocated by buddy-allocator:
> >       /proc/vmstat:nr_memmap * PAGE_SIZE
> >
> > Accounting total Perpage metadata allocated on the machine:
> >       (/proc/vmstat:nr_memmap_boot +
> >        /proc/vmstat:nr_memmap) * PAGE_SIZE
> >
> > Utility for userspace:
> >
> > Observability: Describe the amount of memory overhead that is
> > going to per-page metadata on the system at any given time since
> > this overhead is not currently observable.
> >
> > Debugging: Tracking the changes or absolute value in struct pages
> > can help detect anomalies as they can be correlated with other
> > metrics in the machine (e.g., memtotal, number of huge pages,
> > etc).
> >
> > page_ext overheads: Some kernel features such as page_owner
> > page_table_check that use page_ext can be optionally enabled via
> > kernel parameters. Having the total per-page metadata information
> > helps users precisely measure impact. Furthermore, page-metadata
> > metrics will reflect the amount of struct pages reliquished
> > (or overhead reduced) when hugetlbfs pages are reserved which
> > will vary depending on whether hugetlb vmemmap optimization is
> > enabled or not.
> >
> > For background and results see:
> > lore.kernel.org/all/20240220214558.3377482-1-souravpanda@...gle.com
> >
> > Acked-by: David Rientjes <rientjes@...gle.com>
> > Signed-off-by: Sourav Panda <souravpanda@...gle.com>
> > Reviewed-by: Pasha Tatashin <pasha.tatashin@...een.com>
>
> This patch is leading to Oops in 6.11-rc1 when CONFIG_MEMORY_HOTPLUG
> is enabled. Folks hitting it have had success with reverting this patch.
> Disabling CONFIG_MEMORY_HOTPLUG is not a long term solution.
>
> Reported here:
> https://lore.kernel.org/linux-cxl/CAHj4cs9Ax1=CoJkgBGP_+sNu6-6=6v=_L-ZBZY0bVLD3wUWZQg@mail.gmail.com/

Thank you for the heads up. Can you please attach a full config file,
also was anyone able to reproduce this problem in qemu with emulated
nvdimm?

Pasha