linux-kernel - Re: [PATCH] mm, vmstat: Allow WQ concurrency to discover memory reclaim doesn't make any progress

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 25 Nov 2015 12:07:05 +0100
From:	Michal Hocko <mhocko@...nel.org>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
	Tejun Heo <tj@...nel.org>,
	Cristopher Lameter <clameter@....com>,
	Arkadiusz Miśkiewicz <arekm@...en.pl>,
	linux-mm@...ck.org, LKML <linux-kernel@...r.kernel.org>,
	Joonsoo Kim <js1304@...il.com>,
	Christoph Lameter <cl@...ux.com>
Subject: Re: [PATCH] mm, vmstat: Allow WQ concurrency to discover memory
 reclaim doesn't make any progress

On Tue 24-11-15 15:44:48, Andrew Morton wrote:
[...]
> > Even though we haven't seen bug reports in the past I would suggest
> > backporting this to the stable trees. The issue is present since we have
> > stopped useing congestion_wait in the retry loop because WQ concurrency
> > is older as well as vmstat worqueue based refresh AFAICS.
>
> hm, I'm reluctant.  If the patch fixes something that real people are
> really hurting from then yes.  But I suspect this is just one fly-swat
> amongst many.

Arkadiusz was seeing reclaim issues [1] on 4.1 kernel. I didn't have
time to look deeper in that report but vmstat counters seemed terribly
outdated and the issue went away when this patch was used. The thing is
that there were others in the bundle so it is not 100% clear whether the
patch alone helped or it was just a part of the puzzle.

Anyway I think that the issue is not solely theoretical. WQ_MEM_RECLAIM
is simply not working if the allocation path doesn't sleep currently and
my understanding of what Tejun claims [2] is that that reimplementing WQ
concurrency would be too intrusive and lacks sufficient justification
because other kernel paths do sleep. This patch tries to reduce the
sleep only to worker threads which should not cause any problems to
regular tasks.

I am open to any other suggestions. I do not like artificial sleep as
well but this sounds like the most practical way to go now.

[1] http://lkml.kernel.org/r/201511102313.36685.arekm@maven.pl
[2] http://lkml.kernel.org/r/20151106001648.GA18183@mtj.duckdns.org
-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/