linux-kernel - Re: [PATCH 0/3] Make workingset detection logic memcg aware

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150806085911.GL11971@esperanza>
Date:	Thu, 6 Aug 2015 11:59:11 +0300
From:	Vladimir Davydov <vdavydov@...allels.com>
To:	Kamezawa Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
CC:	Andrew Morton <akpm@...ux-foundation.org>,
	Johannes Weiner <hannes@...xchg.org>,
	Michal Hocko <mhocko@...nel.org>,
	Minchan Kim <minchan@...nel.org>,
	Rik van Riel <riel@...hat.com>, Mel Gorman <mgorman@...e.de>,
	<linux-mm@...ck.org>, <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 0/3] Make workingset detection logic memcg aware

On Wed, Aug 05, 2015 at 10:34:58AM +0900, Kamezawa Hiroyuki wrote:

> Reading discussion, I feel storing more data is difficult, too.

Yep, even with the current 16-bit memcg id. Things would get even worse
if we wanted to extend it one day (will we?)

> 
> I wonder, rather than collecting more data, rough calculation can help the situation.
> for example,
> 
>    (refault_disatance calculated in zone) * memcg_reclaim_ratio < memcg's active list
> 
> If one of per-zone calc or per-memcg calc returns true, refault should be true.
> 
> memcg_reclaim_ratio is the percentage of scan in a memcg against in a zone.

This particular formula wouldn't work I'm afraid. If there are two
isolated cgroups issuing local reclaim on the same zone, the refault
distance needed for activation would be reduced by half for no apparent
reason.

The thing is that there is no need in inventing anything if refaults
from different cgroups are infrequent - it is enough to store only
zone/node ids in shadow entries then, as this patch set does. The
question remains, can we disregard them? Sometimes we need to sacrifice
accuracy for the sake of performance and/or code simplicity. E.g.
inter-cgroup concurrent file writes are not supported in the
implementation of the blkio writeback accounting AFAIK. May be, we could
neglect inter-cgroup refaults too? My point is that even if two cgroups
are actively sharing the same file, its pages will end up in the cgroup
which experiences less memory pressure (most likely, the one with the
greater limit), so inter-cgroup refaults should be rare. Am I wrong?

Anyway, workingset detection is broken for local reclaim (activations
are random) and needs to be fixed. What is worse, shadow entries are
accounted per memcg, but reclaimed only on global memory pressure, so
that they can eat all RAM available to a container w/o giving it a
chance to reclaim it. That said, even this patch set is a huge step
forward, because it makes activations much more deterministic and fixes
per memcg shadow nodes reclaim.

Thanks,
Vladimir
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/