linux-kernel - Re: Detecting page cache trashing state

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170918163434.GA11236@cmpxchg.org>
Date:   Mon, 18 Sep 2017 09:34:34 -0700
From:   Johannes Weiner <hannes@...xchg.org>
To:     Taras Kondratiuk <takondra@...co.com>
Cc:     Michal Hocko <mhocko@...nel.org>, linux-mm@...ck.org,
        xe-linux-external@...co.com,
        Ruslan Ruslichenko <rruslich@...co.com>,
        linux-kernel@...r.kernel.org
Subject: Re: Detecting page cache trashing state

Hi Taras,

On Fri, Sep 15, 2017 at 10:28:30AM -0700, Taras Kondratiuk wrote:
> Quoting Michal Hocko (2017-09-15 07:36:19)
> > On Thu 14-09-17 17:16:27, Taras Kondratiuk wrote:
> > > Has somebody faced similar issue? How are you solving it?
> > 
> > Yes this is a pain point for a _long_ time. And we still do not have a
> > good answer upstream. Johannes has been playing in this area [1].
> > The main problem is that our OOM detection logic is based on the ability
> > to reclaim memory to allocate new memory. And that is pretty much true
> > for the pagecache when you are trashing. So we do not know that
> > basically whole time is spent refaulting the memory back and forth.
> > We do have some refault stats for the page cache but that is not
> > integrated to the oom detection logic because this is really a
> > non-trivial problem to solve without triggering early oom killer
> > invocations.
> > 
> > [1] http://lkml.kernel.org/r/20170727153010.23347-1-hannes@cmpxchg.org
> 
> Thanks Michal. memdelay looks promising. We will check it.

Great, I'm obviously interested in more users of it :) Please find
attached the latest version of the patch series based on v4.13.

It needs a bit more refactoring in the scheduler bits before
resubmission, but it already contains a couple of fixes and
improvements since the first version I sent out.

Let me know if you need help rebasing to a different kernel version.

View attachment "0001-sched-loadavg-consolidate-LOAD_INT-LOAD_FRAC-macros.patch" of type "text/x-diff" (3932 bytes)

View attachment "0002-mm-workingset-tell-cache-transitions-from-workingset.patch" of type "text/x-diff" (14639 bytes)

View attachment "0003-mm-sched-memdelay-memory-health-interface-for-system.patch" of type "text/x-diff" (35404 bytes)