linux-kernel - Re: [LSF/MM/BPF TOPIC] DAMON Requirements for Access-aware MM of Future

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAJj2-QF1y1XNRYy9sT5c5=pi20+67Duf9B2Amatnu01HLmw06Q@mail.gmail.com>
Date: Wed, 29 Jan 2025 18:15:08 -0800
From: Yuanchu Xie <yuanchu@...gle.com>
To: Gregory Price <gourry@...rry.net>
Cc: SeongJae Park <sj@...nel.org>, lsf-pc@...ts.linux-foundation.org, damon@...ts.linux.dev, 
	linux-mm@...ck.org, linux-kernel@...r.kernel.org, kernel-team@...a.com, 
	Raghavendra K T <raghavendra.kt@....com>, Jonathan Cameron <Jonathan.Cameron@...wei.com>, 
	Kaiyang Zhao <kaiyang2@...cmu.edu>, Jiaming Yan <jiamingy@...zon.com>, 
	Honggyu Kim <honggyu.kim@...com>
Subject: Re: [LSF/MM/BPF TOPIC] DAMON Requirements for Access-aware MM of Future

On Mon, Jan 13, 2025 at 7:06 PM Gregory Price <gourry@...rry.net> wrote:
> 5) Scarce resources
>
>    We need to be careful not to consume excessive amounts of resources
>    in an attempt to track all these identifying mechanisms.  Even 1 byte
>    per folio is 256MB on a 1TB machine.  This gets out of hand quick.
>
>    With task-work, I was able to add no additional resource consumption,
>    but deferring to a fully async scenario and needing to track things
>    like last-accessing CPU, timestamps, and etc.
>
>    We'll need to examine this closely if we decide to aggregate either
>    of these mechanisms.
My concern with physical address space monitoring is fragmentation. I
ran some numbers on a few prod machines. Grouping by regions with the
same memcg and ignoring any unmapped memory to be generous, machines
with higher utilization can have a region/total pages ratio of ~40%,
and even those with lower utilization (<50%) can also reach 20%.
Accurately tracking these regions would require quite the region
metadata, on the order of GBs.