linux-kernel - Re: [PATCH 0/4] mm/damon: introduce DAMON_STAT for simple and practical access monitoring

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <20250529170052.37757-1-sj@kernel.org>
Date: Thu, 29 May 2025 10:00:52 -0700
From: SeongJae Park <sj@...nel.org>
To: Honggyu Kim <honggyu.kim@...com>
Cc: SeongJae Park <sj@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	kernel_team@...ynix.com,
	Jonathan Corbet <corbet@....net>,
	damon@...ts.linux.dev,
	kernel-team@...a.com,
	linux-doc@...r.kernel.org,
	linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: [PATCH 0/4] mm/damon: introduce DAMON_STAT for simple and practical access monitoring

Hi Honggyu,

On Thu, 29 May 2025 16:21:39 +0900 Honggyu Kim <honggyu.kim@...com> wrote:

> Hi SeongJae,
> 
> Thanks for your work.
> 
> On 5/27/2025 6:09 AM, SeongJae Park wrote:
> > DAMON-based access monitoring is not simple due to required DAMON
> > control and results visualizations.  Introduce a static kernel module
> > for making it simple.  The module can be enabled without manual setup
> > and provides access pattern metrics that easy to fetch and understand
> > the practical access pattern information, namely estimated memory
> > bandwidth and memory idle time percentiles.
> > 
> > Background and Problems
> > =======================
> > 
> > DAMON can be used for monitoring data access patterns of the system and
> > workloads.  Specifically, users can start DAMON to monitor access events
> > on specific address space with fine controls including address ranges to
> > monitor and time intervals between samplings and aggregations.  The
> > resulting access information snapshot contains access frequency
> > (nr_accesses) and how long the frequency was kept (age) for each byte.
> > 
> > The monitoring usage is not simple and practical enough for production
> > usage.  Users should first start DAMON with a number of parameters, and
> > wait until DAMON's monitoring results capture a reasonable amount of the
> > time data (age).  In production, such manual start and wait is
> > impractical to capture useful information from a high number of machines
> > in a timely manner.
> > 
> > The monitoring result is also too detailed to be used on production
> > environments.  The raw results are hard to be aggregated and/or compared
> > for production environments having a large scale of time, space and
> > machines fleet.
> > 
> > Users have to implement and use their own automation of DAMON control
> > and results processing.  It is repetitive and challenging since there is
> > no good reference or guideline for such automation.
> > 
> > Solution: DAMON_STAT
> > ====================
> > 
> > Implement such automation in kernel space as a static kernel module,
> > namely DAMON_STAT.  It can be enabled at build, boot, or run time via
> > its build configuration or module parameter.  It monitors the entire
> > physical address space with monitoring intervals that auto-tuned for a
> > reasonable amount of access observations and minimum overhead.  It
> > converts the raw monitoring results into simpler metrics that can easily
> > be aggregated and compared, namely estimated memory bandwidth and idle
> > time percentiles.  Refer to the commit messages of the second and the
> > third patches of this patch series for more details about the metrics.
> 
> I see the description looks good but it'd be useful if you could share some
> execution commands and expected output examples that some newbies can get better
> ideas.

Thank you for the feedback.  In my humble opinion, nevertheless, this is what
the fourth patch of this series is aiming to provide.  And thanks to your
comment, I now realize I forgot mentioning this on the above comment.

Of course we can add execution commands and output example here, but without
understanding of the metrics, it would just look like mysterious numbers.
Meanhile, user interface of the module is simply module parameters that should
be familiar to most kernel users, so I don't think it necessarily deserves
example commands.

> 
> I honestly do not have a clear idea how I can use this kind of static kernel
> modules as general users although I have developed some features of DAMON.
> 
> So could you please help?

I agree this cover letter is not enough for getting clear idea of how readers
can use this feature.  Thank you for letting me realize this.  But I think this
cover letter might be better to keep the brevity, as long as it points where
readers can get necessary details.  What do you think about revising this cover
letter to

1. point the fourth patch for usage of DAMON_STAT, and
2. elaborate the fact that readers are recommended to read the metric
   definitions and usages to get more clear idea of DAMON_STAT?


Thanks,
SJ

[...]