linux-kernel - Re: [RFC] mm: memcg: add priority for soft limit reclaiming

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190924133016.GT23050@dhcp22.suse.cz>
Date:   Tue, 24 Sep 2019 15:30:16 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Hillf Danton <hdanton@...a.com>
Cc:     Johannes Weiner <hannes@...xchg.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        linux <linux-kernel@...r.kernel.org>,
        linux-mm <linux-mm@...ck.org>,
        Shakeel Butt <shakeelb@...gle.com>,
        Roman Gushchin <guro@...com>,
        Matthew Wilcox <willy@...radead.org>
Subject: Re: [RFC] mm: memcg: add priority for soft limit reclaiming

On Tue 24-09-19 15:36:42, Hillf Danton wrote:
> 
> On Mon, 23 Sep 2019 21:28:34 Michal Hocko wrote:
> > 
> > On Mon 23-09-19 21:04:59, Hillf Danton wrote:
> > >
> > > On Thu, 19 Sep 2019 21:32:31 +0800 Michal Hocko wrote:
> > > >
> > > > On Thu 19-09-19 21:13:32, Hillf Danton wrote:
> > > > >
> > > > > Currently memory controler is playing increasingly important role in
> > > > > how memory is used and how pages are reclaimed on memory pressure.
> > > > >
> > > > > In daily works memcg is often created for critical tasks and their pre
> > > > > configured memory usage is supposed to be met even on memory pressure.
> > > > > Administrator wants to make it configurable that the pages consumed by
> > > > > memcg-B can be reclaimed by page allocations invoked not by memcg-A but
> > > > > by memcg-C.
> > > >
> > > > I am not really sure I understand the usecase well but this sounds like
> > > > what memory reclaim protection in v2 is aiming at.
> > > >
> > Please describe the usecase.
> > 
> It is for quite a while that task-A has been able to preempt task-B for
> cpu cycles. IOW the physical resource cpu cycles are preemptible.
> 
> Are physical pages are preemptible too in the same manner?
> Nope without priority defined for pages currently (say the link between
> page->nice and task->nice).
> 
> The slrp is added for memcg instead of nice because 1) it is only used
> in the page reclaiming context (in memcg it is soft limit reclaiming),
> and 2) it is difficult to compare reclaimer and reclaimee task->nice
> directly in that context as only info about reclaimer and lru page is
> available.
> 
> Here task->nice is replaced with memcg->slrp in order to do page
> preemption, PP. There is no way for task-A to PP task-B, but the
> group containing task-A can PP the group containing task-B.
> That preemption needs code within 100 lines as you see on top of
> the current memory controller framework.

This is exactly what the reclaim protection in memcg v2 is meant to be
used for. Also soft limit reclaim is absolutely terrible to achieve that
because it is just too gross to result in any smooth experience (just
have a look how it is doing priority 0 scannig!).

I am not going to even go further wrt the implementation because I
belive the priority is even semantically broken wrt hierarchical
behavior.

But really, make sure you look into the existing feature set that memcg
v2 provides already and come back if you find it unsuitable and we can
move from there. Soft limit reclaim is dead and we should let it RIP.
-- 
Michal Hocko
SUSE Labs