Date:   Thu, 17 Feb 2022 16:19:34 +0000
From:   SeongJae Park <sj@...nel.org>
To:     akpm@...ux-foundation.org
Cc:     rientjes@...gle.com, xhao@...ux.alibaba.com,
        linux-damon@...zon.com, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, SeongJae Park <sj@...nel.org>
Subject: [RFC PATCH 0/4] Introduce DAMON sysfs interface

DAMON's debugfs-based user interface served very well, so far.  However, it
unnecessarily depends on debugfs, while DAMON is not aimed to be used for only
debugging.  Also, the interface receives multiple values via one file.  For
example, schemes file receives 18 values.  As a result, it is not only hard to
be used, but also difficult to be extended.  Especially, keeping backward
compatibility of user space tools is getting only challenging.  It would be
better to implement another reliable and flexible interface and deprecate the
debugfs interface in long term.

For the reason, this patchset introduces a sysfs-based new user interface of
DAMON.  The idea of the new interface is, using directory hierarchies and
making one file for one value.  For a short example, users can do the virtual
address monitoring via the interface as below:

    # cd /sys/kernel/mm/damon/admin/
    # echo 1 > kdamonds/nr
    # echo 1 > kdamonds/0/contexts/nr
    # echo vaddr > kdamonds/0/contexts/0/damon_type
    # echo 1 > kdamonds/0/contexts/0/targets/nr
    # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid
    # echo on > kdamonds/0/state

Changes that will be made for dropping RFC tag
----------------------------------------------

As the RFC tag means, this is not for merged in the mainline but only for early
comments.  Therefore, this version is not implementing all the features of
DAMON debugfs interface but only basic virtual address space monitoring.  The
official version of the patchset will provide all the features that DAMON
debugfs interface is providing.  Also, this RFC patchset lacks formal
documentation.  It will be a part of the official version.

Future plan of DAMON debugfs interface deprecation
--------------------------------------------------

Once the official version of this patchset is merged, DAMON debugfs interface
development will be frozen.  That is, we will keep the interface works as is
now, but it will not provide any new feature of DAMON.  The support will be
continued only until next LTS release.  After that, we will drop DAMON debugfs
interface.  Changes to the documentation for explicitly announcing the
deprecation plan would be a part of the official version of this patchset.

Main difference between DAMON_DBGFS and DAMON_SYSFS
---------------------------------------------------

DAMON debugfs interface allows multiple monitoring contexts, but it asks users
to turn those all on and off at once.  It's not a big problem but makes the
operation a little bit complex.  DAMON_SYSFS allows users to turn on and off
monitoring contexts individually.

The Hierarchy
=============

In a glance, the files hierarchy of the sysfs interface is as below.

    /sys/kernel/mm/damon/admin
    │ kdamonds
    │ │ nr
    │ │ 0/
    │ │ │ state,pid
    │ │ │ contexts
    │ │ │ │ nr
    │ │ │ │ 0/
    │ │ │ │ │ damon_type
    │ │ │ │ │ monitoring_attrs/
    │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
    │ │ │ │ │ │ nr_regions/min,max
    │ │ │ │ │ targets/
    │ │ │ │ │ │ nr
    │ │ │ │ │ │ 0/
    │ │ │ │ │ │ │ pid
    │ │ │ │ │ │ │ regions/
    │ │ │ │ │ │ │ │ nr
    │ │ │ │ │ │ │ │ 0/
    │ │ │ │ │ │ │ │ │ start,end
    │ │ │ │ │ │ │ │ ...
    │ │ │ │ │ │ ...
    │ │ │ │ │ ...
    │ │ ...

Root
----

The root of the DAMON sysfs is /sys/kernel/mm/damon/, and it has one directory
named 'admin'.  The directory contains the interface for privileged user space
programs.  User space tools or deamons having root permission could use this
directory.  In a future, sibling directories for non-root user space tools or
deamons (we could allow them to do monitong of their own virtual address space)
or control of in-kernel DAMON-based deamons could be created.

kdamonds/
---------

The monitoring-related information including request specifications and results
are called DAMON context.  DAMON executes a set of the contexts with a kernel
thread called kdamond (for now, only one context per kdamond is supported), and
multiple kdamonds could run in parallel.  This directory has files for
controlling the kdamonds.

In the beginning, this directory has only one file, 'nr'.  Writing a number
(`N`) to the file creates the number of child directories named `0` to `N-1`.
Each directory represents each kdamond.

kdamonds/<N>/
-------------

In each kdamond directory, two files (`state` and `pid`) and one directory
(`contexts`) reside.

Reading `state` returns `on` if the kdamond is currently running, or `off` if
it is not running.  Writing `on` or `off` makes the kdamond be in the state.
If the state is `on`, reading `pid` shows the pid of the kdamond thread.

`contexts` directory resembles `kdamonds`.  It contains files for controlling
the monitoring contexts that this kdamond will execute.

kdamonds/<N>/contexts/
----------------------

In the beginning, this directory has only one file, 'nr'.  Writing a number
(`N`) to the file creates the number of child directories named as `0` to
`N-1`.  Each directory represents each monitoring context.  At the moment, only
one context per kdamond is supported, so only `0` or `1` can be written to the
file.

contexts/<N>/
-------------

In each context directory, one file (`operations`) and two directories
(`monitoring_attrs` and `targets`) reside.

DAMON supports multiple types of monitoring operations, including those for
virtual address space and physical address space.  You can set and show what
type of monitoring operations you want to use with the context by writing one
of below keywords to, and reading the file.

- vaddr: Monitor virtual address spaces of specific processes

Files for specifying attributes of the monitoring including required quality
and efficiency of the monitoring are in `monitoring_attrs` directory, while
files for specifying to what memory regions the monitoring should be done are
in `targets` directory.

contexts/<N>/monitoring_attrs/
------------------------------

In this directory, you can show two directories, `intervals` and `nr_regions`.

Under `intervals` directory, three files for DAMON's sampling interval
(`sample_us`), aggregation interval (`aggr_us`) and update interval
(`update_us`) exist.  You can set and get the values by writing to and reading
from the files.

Under `nr_regions` directory, two files for the lower-bound and upper-bound of
DAMON's monitoring regions (`min` and `max`, respectively), which controls the
monitoring overhead, reside.  You can set and get the values by writing to and
rading from the files.

For more details about the intervals and monitoring regions range, please read
the Design document[1].

[1] https://docs.kernel.org/vm/damon/design.html

contexts/<N>/targets/
---------------------

In the beginning, this directory has only one file, 'nr'.  Writing a number
(`N`) to the file creates the number of child directories named `0` to `N-1`.
Each directory represents each monitoring target.

targets/<N>/
------------

In each target directory, one file (`pid`) exists.

You can make the context to monitor the virtual address space of a process by
writing the pid of the process to the file, and show what process's virtual
address space the context is set to monitor by reading the file.

SeongJae Park (4):
  mm/damon: Implement a sysfs-based DAMON user interface
  mm/damon/core: Allow non-exclusive DAMON start/stop
  mm/damon/sysfs: Link DAMON to 'state' file read/write functions
  selftests/damon: Add a test for DAMON sysfs interface

 include/linux/damon.h                  |    2 +-
 mm/damon/Kconfig                       |    7 +
 mm/damon/Makefile                      |    1 +
 mm/damon/core.c                        |   22 +-
 mm/damon/dbgfs.c                       |    2 +-
 mm/damon/reclaim.c                     |    2 +-
 mm/damon/sysfs.c                       | 1258 ++++++++++++++++++++++++
 tools/testing/selftests/damon/Makefile |    1 +
 tools/testing/selftests/damon/sysfs.sh |  200 ++++
 9 files changed, 1486 insertions(+), 9 deletions(-)
 create mode 100644 mm/damon/sysfs.c
 create mode 100755 tools/testing/selftests/damon/sysfs.sh

-- 
2.17.1

Powered by blists - more mailing lists