Message-ID: <82ea9158-c71e-4c11-a46d-397e9ec9349b@sk.com>
Date: Thu, 29 May 2025 16:21:39 +0900
From: Honggyu Kim <honggyu.kim@...com>
To: SeongJae Park <sj@...nel.org>, Andrew Morton <akpm@...ux-foundation.org>
Cc: kernel_team@...ynix.com, Jonathan Corbet <corbet@....net>,
 damon@...ts.linux.dev, kernel-team@...a.com, linux-doc@...r.kernel.org,
 linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: [PATCH 0/4] mm/damon: introduce DAMON_STAT for simple and
 practical access monitoring

Hi SeongJae,

Thanks for your work.

On 5/27/2025 6:09 AM, SeongJae Park wrote:
> DAMON-based access monitoring is not simple due to required DAMON
> control and results visualizations.  Introduce a static kernel module
> for making it simple.  The module can be enabled without manual setup
> and provides access pattern metrics that easy to fetch and understand
> the practical access pattern information, namely estimated memory
> bandwidth and memory idle time percentiles.
> 
> Background and Problems
> =======================
> 
> DAMON can be used for monitoring data access patterns of the system and
> workloads.  Specifically, users can start DAMON to monitor access events
> on specific address space with fine controls including address ranges to
> monitor and time intervals between samplings and aggregations.  The
> resulting access information snapshot contains access frequency
> (nr_accesses) and how long the frequency was kept (age) for each byte.
> 
> The monitoring usage is not simple and practical enough for production
> usage.  Users should first start DAMON with a number of parameters, and
> wait until DAMON's monitoring results capture a reasonable amount of the
> time data (age).  In production, such manual start and wait is
> impractical to capture useful information from a high number of machines
> in a timely manner.
> 
> The monitoring result is also too detailed to be used on production
> environments.  The raw results are hard to be aggregated and/or compared
> for production environments having a large scale of time, space and
> machines fleet.
> 
> Users have to implement and use their own automation of DAMON control
> and results processing.  It is repetitive and challenging since there is
> no good reference or guideline for such automation.
> 
> Solution: DAMON_STAT
> ====================
> 
> Implement such automation in kernel space as a static kernel module,
> namely DAMON_STAT.  It can be enabled at build, boot, or run time via
> its build configuration or module parameter.  It monitors the entire
> physical address space with monitoring intervals that auto-tuned for a
> reasonable amount of access observations and minimum overhead.  It
> converts the raw monitoring results into simpler metrics that can easily
> be aggregated and compared, namely estimated memory bandwidth and idle
> time percentiles.  Refer to the commit messages of the second and the
> third patches of this patch series for more details about the metrics.

I see the description looks good but it'd be useful if you could share some
execution commands and expected output examples that some newbies can get better
ideas.

I honestly do not have a clear idea how I can use this kind of static kernel
modules as general users although I have developed some features of DAMON.

So could you please help?

Thanks,
Honggyu

> 
> Discussions
> ===========
> 
> The module aims to be useful on production environments constructed with
> a large number of machines that run a long time.  The auto-tuned
> monitoring intervals ensure a reasonable quality of the outputs.  The
> auto-tuning also ensures its overhead be reasonable and low enough to be
> enabled always on the production.  The simplified monitoring results
> metrics can be useful for showing both coldness (idle time percentiles)
> and hotness (memory bandwidth) of the system's access pattern.  We
> expect the information can be useful for assessing system memory
> utilization and inspiring optimizations or investigations on both kernel
> and user space memory management logics for large scale fleets.
> 
> We hence expect the module is good enough to be just used in most
> environments.  For special cases that require a custom access monitoring
> automation, users will still benefit by using DAMON_STAT as a reference
> or a guideline for their specialized automation.
> 
> Revision History
> ================
> 
> Changes from RFC
> (https://lore.kernel.org/20250519164415.43935-1-sj@kernel.org)
> - Add an admin-guide documentation
> - Wordsmith commit messages
> - Rebase to latest mm-new
> 
> SeongJae Park (4):
>    mm/damon: introduce DAMON_STAT module
>    mm/damon/stat: calculate and expose estimated memory bandwidth
>    mm/damon/stat: calculate and expose idle time percentiles
>    Docs/admin-guide/mm/damon: add DAMON_STAT usage document
> 
>   Documentation/admin-guide/mm/damon/index.rst |   1 +
>   Documentation/admin-guide/mm/damon/stat.rst  |  69 ++++++
>   mm/damon/Kconfig                             |  16 ++
>   mm/damon/Makefile                            |   1 +
>   mm/damon/stat.c                              | 245 +++++++++++++++++++
>   5 files changed, 332 insertions(+)
>   create mode 100644 Documentation/admin-guide/mm/damon/stat.rst
>   create mode 100644 mm/damon/stat.c
> 
> 
> base-commit: 90887f57d7a67917136e7c70d26fb3f2fcdc6f53

Powered by blists - more mailing lists