linux-kernel - Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CALWz4iwa=d_CDgUmsWB=xaS-7-X+r+2riD7HM3yxHWRTyResOg@mail.gmail.com>
Date:	Tue, 24 Jan 2012 15:50:40 -0800
From:	Ying Han <yinghan@...gle.com>
To:	Hillf Danton <dhillf@...il.com>
Cc:	Michal Hocko <mhocko@...e.cz>, linux-mm@...ck.org,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Hugh Dickins <hughd@...gle.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Johannes Weiner <hannes@...xchg.org>
Subject: Re: [PATCH] mm: memcg: fix over reclaiming mem cgroup

On Mon, Jan 23, 2012 at 7:26 PM, Hillf Danton <dhillf@...il.com> wrote:
> Hi all
>
> On Tue, Jan 24, 2012 at 3:14 AM, Ying Han <yinghan@...gle.com> wrote:
>> On Mon, Jan 23, 2012 at 5:02 AM, Michal Hocko <mhocko@...e.cz> wrote:
>>> On Sat 21-01-12 22:49:23, Hillf Danton wrote:
>>>> In soft limit reclaim, overreclaim occurs when pages are reclaimed from mem
>>>> group that is under its soft limit, or when more pages are reclaimd than the
>>>> exceeding amount, then performance of reclaimee goes down accordingly.
>>>
>>> First of all soft reclaim is more a help for the global memory pressure
>>> balancing rather than any guarantee about how much we reclaim for the
>>> group.
>>> We need to do more changes in order to make it a guarantee.
>>> For example you implementation will cause severe problems when all
>>> cgroups are soft unlimited (default conf.) or when nobody is above the
>>> limit but the total consumption triggers the global reclaim. Therefore
>>> nobody is in excess and you would skip all groups and only bang on the
>>> root memcg.
>
> If soft limits are set to be limited and there are no excessors,
> who are consuming physical pages? The consumers maybe those with soft
> unlimited. If so, they should be punished first, based on the assumption that
> the unlimited is treated with no guarantee. Then soft limit guarantee could
> be assured without changes in the current default setting of soft limit, no?

The current code set the softlimit default to be RESOURCE_MAX, but we
do have plan to change it to 0. Then every cgroup will be eligible for
soft limit reclaim unless we set it otherwise. Not sure if that will
answer some of the questions?

>
> With soft limit available, victims are only selected from excessors, I think.
>
>>>
>>> Ying Han has a patch which basically skips all cgroups which are under
>>> its limit until we reach a certain reclaim priority but even for this we
>>> need some additional changes - e.g. reverse the current default setting
>>> of the soft limit.
>>>
>>> Anyway, I like the nr_to_reclaim reduction idea because we have to do
>>> this in some way because the global reclaim starts with ULONG
>>> nr_to_scan.
>>
>> Agree with Michal where there are quite a lot changes we need to get
>> in for soft limit before any further optimization.
>>
>> Hillf, please refer to the patch from Johannes
>> https://lkml.org/lkml/2012/1/13/99 which got quite a lot recent
>> discussions. I am expecting to get that in before further soft limit
>> changes.
>>
>
> Johannes did great cleanup, why barriered?

Not sure about the barriered here. As far as I can tell, some changes
we are talking about here would be easier to be applied after that
cleanup.

--Ying
>
> Thanks
> Hillf
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/