linux-kernel - Re: WARNINGs in set_task_reclaim_state with memory cgroup and full memory usage

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 27 Aug 2019 14:55:21 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Yafang Shao <laoar.shao@...il.com>
Cc:     Yang Shi <yang.shi@...ux.alibaba.com>,
        Adric Blake <promarbler14@...il.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Kirill Tkhai <ktkhai@...tuozzo.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Daniel Jordan <daniel.m.jordan@...cle.com>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Linux MM <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: WARNINGs in set_task_reclaim_state with memory cgroup and full
 memory usage

On Tue 27-08-19 20:19:34, Yafang Shao wrote:
> On Tue, Aug 27, 2019 at 8:03 PM Michal Hocko <mhocko@...nel.org> wrote:
> >
> > On Tue 27-08-19 19:56:16, Yafang Shao wrote:
> > > On Tue, Aug 27, 2019 at 7:50 PM Michal Hocko <mhocko@...nel.org> wrote:
> > > >
> > > > On Tue 27-08-19 19:43:49, Yafang Shao wrote:
> > > > > On Tue, Aug 27, 2019 at 6:43 PM Michal Hocko <mhocko@...nel.org> wrote:
> > > > > >
> > > > > > If there are no objection to the patch I will post it as a standalong
> > > > > > one.
> > > > >
> > > > > I have no objection to your patch. It could fix the issue.
> > > > >
> > > > > I still think that it is not proper to use a new scan_control here as
> > > > > it breaks the global reclaim context.
> > > > >
> > > > > This context switch from global reclaim to memcg reclaim is very
> > > > > subtle change to the subsequent processing, that may cause some
> > > > > unexpected behavior.
> > > >
> > > > Why would it break it? Could you be more specific please?
> > > >
> > >
> > > Hmm, I have explained it when replying to  Hillf's patch.
> > > The most suspcious one is settting target_mem_cgroup here, because we
> > > only use it to judge whether it is in global reclaim.
> > > While the memcg softlimit reclaim is really in global reclaims.
> >
> > But we are reclaim the target_mem_cgroup hierarchy. This is the whole
> > point of the soft reclaim. Push down that hierarchy below the configured
> > limit. And that is why we absolutely have to switch the reclaim context.
> >
> 
> One obvious issue is the reclaim couters may not correct.
> See shrink_inactive_list().
> The pages relcaimed in memcg softlimit will not be counted to
> PGSCAN_{DIRECT, KSWAPD} and PGSTEAL_{DIRECT, KSWAPD}.

And again this a long term behavior so I would be really curious why it
is considered a bug now. Really, the semantic of the soft limit is
weird. It has been grafted into the reclaim while it was doing a
semantically different thing. It doesn't really reflect kswapd or direct
reclaim targets.
-- 
Michal Hocko
SUSE Labs