lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Fri, 11 Jun 2021 17:44:18 +0000
From:   "Boehme, Markus" <markubo@...zon.de>
To:     "sj38.park@...il.com" <sj38.park@...il.com>,
        "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>
CC:     "benh@...nel.crashing.org" <benh@...nel.crashing.org>,
        "Foerster, Leonard" <foersleo@...zon.de>,
        "guoju.fgj@...baba-inc.com" <guoju.fgj@...baba-inc.com>,
        "brendanhiggins@...gle.com" <brendanhiggins@...gle.com>,
        "riel@...riel.com" <riel@...riel.com>,
        "greg@...ah.com" <greg@...ah.com>,
        "linux-damon@...zon.com" <linux-damon@...zon.com>,
        "david@...hat.com" <david@...hat.com>,
        "rientjes@...gle.com" <rientjes@...gle.com>,
        "snu@...zon.de" <snu@...zon.de>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "corbet@....net" <corbet@....net>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Park, Seongjae" <sjpark@...zon.de>,
        "Jonathan.Cameron@...wei.com" <Jonathan.Cameron@...wei.com>,
        "namhyung@...nel.org" <namhyung@...nel.org>,
        "rppt@...nel.org" <rppt@...nel.org>,
        "shuah@...nel.org" <shuah@...nel.org>,
        "vbabka@...e.cz" <vbabka@...e.cz>,
        "zgf574564920@...il.com" <zgf574564920@...il.com>,
        "mgorman@...e.de" <mgorman@...e.de>,
        "amit@...nel.org" <amit@...nel.org>,
        "fan.du@...el.com" <fan.du@...el.com>,
        "minchan@...nel.org" <minchan@...nel.org>,
        "rostedt@...dmis.org" <rostedt@...dmis.org>,
        "acme@...nel.org" <acme@...nel.org>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "alexander.shishkin@...ux.intel.com" 
        <alexander.shishkin@...ux.intel.com>,
        "gthelen@...gle.com" <gthelen@...gle.com>,
        "vdavydov.dev@...il.com" <vdavydov.dev@...il.com>,
        "elver@...gle.com" <elver@...gle.com>,
        "shakeelb@...gle.com" <shakeelb@...gle.com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "Woodhouse, David" <dwmw@...zon.co.uk>,
        "linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>
Subject: Re: [PATCH v29 10/13] Documentation: Add documents for DAMON

On Thu, 2021-05-20 at 07:56 +0000, SeongJae Park wrote:
> From: SeongJae Park <sjpark@...zon.de>
> 
> This commit adds documents for DAMON under
> `Documentation/admin-guide/mm/damon/` and `Documentation/vm/damon/`.
> 
> Signed-off-by: SeongJae Park <sjpark@...zon.de>
> ---
>  Documentation/admin-guide/mm/damon/guide.rst | 158 +++++++++++++
>  Documentation/admin-guide/mm/damon/index.rst |  15 ++
>  Documentation/admin-guide/mm/damon/plans.rst |  29 +++
>  Documentation/admin-guide/mm/damon/start.rst | 114 +++++++++
>  Documentation/admin-guide/mm/damon/usage.rst | 112 +++++++++
>  Documentation/admin-guide/mm/index.rst       |   1 +
>  Documentation/vm/damon/api.rst               |  20 ++
>  Documentation/vm/damon/design.rst            | 166 +++++++++++++
>  Documentation/vm/damon/eval.rst              | 232 +++++++++++++++++++
>  Documentation/vm/damon/faq.rst               |  58 +++++
>  Documentation/vm/damon/index.rst             |  31 +++
>  Documentation/vm/index.rst                   |   1 +
>  12 files changed, 937 insertions(+)
>  create mode 100644 Documentation/admin-guide/mm/damon/guide.rst
>  create mode 100644 Documentation/admin-guide/mm/damon/index.rst
>  create mode 100644 Documentation/admin-guide/mm/damon/plans.rst
>  create mode 100644 Documentation/admin-guide/mm/damon/start.rst
>  create mode 100644 Documentation/admin-guide/mm/damon/usage.rst
>  create mode 100644 Documentation/vm/damon/api.rst
>  create mode 100644 Documentation/vm/damon/design.rst
>  create mode 100644 Documentation/vm/damon/eval.rst
>  create mode 100644 Documentation/vm/damon/faq.rst
>  create mode 100644 Documentation/vm/damon/index.rst
> 
> [...]
>
> diff --git a/Documentation/admin-guide/mm/damon/start.rst b/Documentation/admin-guide/mm/damon/start.rst
> new file mode 100644
> index 000000000000..f5bbf1e36836
> --- /dev/null
> +++ b/Documentation/admin-guide/mm/damon/start.rst
> @@ -0,0 +1,114 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +===============
> +Getting Started
> +===============
> +
> +This document briefly describes how you can use DAMON by demonstrating its
> +default user space tool.  Please note that this document describes only a part
> +of its features for brevity.  Please refer to :doc:`usage` for more details.
> [...]
> +
> +
> +Prerequisites
> +=============
> +
> +Kernel
> +------
> +
> +You should first ensure your system is running on a kernel built with
> +``CONFIG_DAMON_*=y``.
> +
> +
> +User Space Tool
> +---------------
> +
> +For the demonstration, we will use the default user space tool for DAMON,
> +called DAMON Operator (DAMO).  It is available at
> +https://github.com/awslabs/damo.  For brevity, below examples assume you set
> +``$PATH`` to point it.  It's not mandatory, though.

"The examples below assume ``damo`` is on your ``$PATH``."?

> +
> +Because DAMO is using the debugfs interface (refer to :doc:`usage` for the
> +detail) of DAMON, you should ensure debugfs is mounted.  Mount it manually as
> +below::
> +
> +    # mount -t debugfs none /sys/kernel/debug/
> +
> +or append below line to your ``/etc/fstab`` file so that your system can
> +automatically mount debugfs from next booting::
> +
> +    debugfs /sys/kernel/debug debugfs defaults 0 0
> +
> +
> [...]
> diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
> new file mode 100644
> index 000000000000..ea3fa6e55f8b
> --- /dev/null
> +++ b/Documentation/admin-guide/mm/damon/usage.rst
> @@ -0,0 +1,112 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +===============
> +Detailed Usages
> +===============
> [...]
> +
> +Tracepoint for Monitoring Results
> +=================================
> +
> +DAMON provides the monitoring results via a tracepoint,
> +``damon:damon_aggregated``.  While the monitoring is turned on, you could
> +record the tracepoint events and show results using tracepoint supporting tools
> +like ``perf``.  For example::
> +
> +    # echo on > monitor_on
> +    # perf record damon:damon_aggregated &

I think that needs to be "-e damon:damon_aggregated".

> +    # sleep 5
> +    # kill 9 $(pidof perf)
> +    # echo off > monitor_on
> +    # perf script
> 
> [...]
>
> diff --git a/Documentation/vm/damon/design.rst b/Documentation/vm/damon/design.rst
> new file mode 100644
> index 000000000000..727d72093f8f
> --- /dev/null
> +++ b/Documentation/vm/damon/design.rst
> @@ -0,0 +1,166 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======
> +Design
> +======
> +
> [...]
> +
> +Reference Implementations of Address Space Specific Primitives
> +==============================================================
> +
> +The low level primitives for the fundamental access monitoring are defined in
> +two parts:
> +
> +1. Identification of the monitoring target address range for the address space.
> +2. Access check of specific address range in the target space.
> +
> +DAMON currently provides the implementation of the primitives for only the
> +virtual address spaces. Below two subsections describe how it works.
> +
> +
> +PTE Accessed-bit Based Access Check
> +-----------------------------------
> +
> +The implementation for the virtual address space uses PTE Accessed-bit for
> +basic access checks.  It finds the relevant PTE Accessed bit from the address
> +by walking the page table for the target task of the address.  In this way, the
> +implementation finds and clears the bit for next sampling target address and
> +checks whether the bit set again after one sampling period.  This could disturb
> +other kernel subsystems using the Accessed bits, namely Idle page tracking and
> +the reclaim logic.  To avoid such disturbances, DAMON makes it mutually
> +exclusive with Idle page tracking and uses ``PG_idle`` and ``PG_young`` page
> +flags to solve the conflict with the reclaim logic, as Idle page tracking does.
> +
> +
> +VMA-based Target Address Range Construction
> +-------------------------------------------
> +
> +Only small parts in the super-huge virtual address space of the processes are
> +mapped to the physical memory and accessed.  Thus, tracking the unmapped
> +address regions is just wasteful.  However, because DAMON can deal with some
> +level of noise using the adaptive regions adjustment mechanism, tracking every
> +mapping is not strictly required but could even incur a high overhead in some
> +cases.  That said, too huge unmapped areas inside the monitoring target should
> +be removed to not take the time for the adaptive mechanism.
> +
> +For the reason, this implementation converts the complex mappings to three
> +distinct regions that cover every mapped area of the address space.  The two
> +gaps between the three regions are the two biggest unmapped areas in the given
> +address space.  The two biggest unmapped areas would be the gap between the
> +heap and the uppermost mmap()-ed region, and the gap between the lowermost
> +mmap()-ed region and the stack in most of the cases.  Because these gaps are
> +exceptionally huge in usual address spaces, excluding these will be sufficient
> +to make a reasonable trade-off.  Below shows this in detail::
> +
> +    <heap>
> +    <BIG UNMAPPED REGION 1>
> +    <uppermost mmap()-ed region>
> +    (small mmap()-ed regions and munmap()-ed regions)
> +    <lowermost mmap()-ed region>
> +    <BIG UNMAPPED REGION 2>
> +    <stack>
> +

Nit: I'd swap these sections so they match the ordered list in the
section overview.

> [...]

I skipped the files you mentioned would be dropped from the next
revision.

Best regards,
Markus



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879


Powered by blists - more mailing lists