linux-kernel - [RFC PATCH 0/3] DAMON: Implement The Data Access Pattern Awared Memory Management Rules

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 10 Feb 2020 16:09:18 +0100
From:   <sjpark@...zon.com>
To:     <akpm@...ux-foundation.org>
CC:     SeongJae Park <sjpark@...zon.de>, <acme@...nel.org>,
        <alexander.shishkin@...ux.intel.com>, <amit@...nel.org>,
        <brendan.d.gregg@...il.com>, <brendanhiggins@...gle.com>,
        <cai@....pw>, <colin.king@...onical.com>, <corbet@....net>,
        <dwmw@...zon.com>, <jolsa@...hat.com>, <kirill@...temov.name>,
        <mark.rutland@....com>, <mgorman@...e.de>, <minchan@...nel.org>,
        <mingo@...hat.com>, <namhyung@...nel.org>, <peterz@...radead.org>,
        <rdunlap@...radead.org>, <rostedt@...dmis.org>,
        <sj38.park@...il.com>, <vdavydov.dev@...il.com>,
        <linux-mm@...ck.org>, <linux-doc@...r.kernel.org>,
        <linux-kernel@...r.kernel.org>
Subject: [RFC PATCH 0/3] DAMON: Implement The Data Access Pattern Awared Memory Management Rules

From: SeongJae Park <sjpark@...zon.de>

DAMON can make data access pattern awared memory management optimizations much
easier.  That said, users who want such optimizations should run DAMON, read
the monitoring results, analyze it, plan a new memory management scheme, and
apply the new scheme by themselves.  It would not be too hard, but still
require some level of efforts.  Such efforts will be really necessary in some
complicated cases.

However, in many other cases, the optimizations would have a simple and common
pattern.  For example, the users would just want the system to apply an actions
to a memory region of a specific size having a specific access frequency for a
specific time.  For example, "page out a memory region larger than 100 MiB but
having a low access frequency more than 10 minutes", or "Use THP for a memory
region larger than 2 MiB having a high access frequency for more than 2
seconds".

This RFC patchset makes DAMON to receive and do such simple optimizations.  All
the things users need to do for such simple cases is only to specify their
requests to DAMON in a form of rules.

For the actions, current implementation supports only a few of ``madvise()``
hints, ``MADV_WILLNEED``, ``MADV_COLD``, ``MADV_PAGEOUT``, ``MADV_HUGEPAGE``,
and ``MADV_NOHUGEPAGE``.

Sequence Of Patches
===================

The first patch allows DAMON to reuse ``madvise()`` code.  The second patch
implements the data access pattern awared memory management rules and its
kernel space programming interface.  Finally, the third patch implements a
debugfs interface for privileged user space people and programs.

The patches are based on the v5.5 plus v4 DAMON patchset[1] and Minchan's
``madvise()`` factoring out patch[2].  Minchan's patch was necessary for reuse
of ``madvise()`` code.  You can also clone the complete git tree:

    $ git clone git://github.com/sjp38/linux -b damon/rules/rfc/v1

The web is also available:
https://github.com/sjp38/linux/releases/tag/damon/rules/rfc/v1

[1] https://lore.kernel.org/linux-mm/20200210144812.26845-1-sjpark@amazon.com/
[2] https://lore.kernel.org/linux-mm/20200128001641.5086-2-minchan@kernel.org/
SeongJae Park (3):
  mm/madvise: Export madvise_common() to mm internal code
  mm/damon/rules: Implement access pattern based management rules
  mm/damon/rules: Implement a debugfs interface

 include/linux/damon.h |  28 ++++
 mm/damon.c            | 317 +++++++++++++++++++++++++++++++++++++++++-
 mm/internal.h         |   4 +
 mm/madvise.c          |   2 +-
 4 files changed, 346 insertions(+), 5 deletions(-)

-- 
2.17.1