| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <4F158418.2090509@gmail.com>
Date: Tue, 17 Jan 2012 22:22:16 +0800
From: Sha <handai.szj@...il.com>
To: Johannes Weiner <hannes@...xchg.org>
CC: Ying Han <yinghan@...gle.com>,
Andrew Morton <akpm@...ux-foundation.org>,
Michal Hocko <mhocko@...e.cz>,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
Balbir Singh <bsingharora@...il.com>, cgroups@...r.kernel.org,
linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [patch 2/2] mm: memcg: hierarchical soft limit reclaim
On 01/14/2012 06:44 AM, Johannes Weiner wrote:
> On Fri, Jan 13, 2012 at 01:31:16PM -0800, Ying Han wrote:
>> On Thu, Jan 12, 2012 at 12:59 AM, Johannes Weiner<hannes@...xchg.org> wrote:
>>> On Wed, Jan 11, 2012 at 01:42:31PM -0800, Ying Han wrote:
>>>> On Tue, Jan 10, 2012 at 7:02 AM, Johannes Weiner<hannes@...xchg.org> wrote:
>>>>> @@ -1318,6 +1123,36 @@ static unsigned long mem_cgroup_margin(struct mem_cgroup *memcg)
>>>>> return margin>> PAGE_SHIFT;
>>>>> }
>>>>>
>>>>> +/**
>>>>> + * mem_cgroup_over_softlimit
>>>>> + * @root: hierarchy root
>>>>> + * @memcg: child of @root to test
>>>>> + *
>>>>> + * Returns %true if @memcg exceeds its own soft limit or contributes
>>>>> + * to the soft limit excess of one of its parents up to and including
>>>>> + * @root.
>>>>> + */
>>>>> +bool mem_cgroup_over_softlimit(struct mem_cgroup *root,
>>>>> + struct mem_cgroup *memcg)
>>>>> +{
>>>>> + if (mem_cgroup_disabled())
>>>>> + return false;
>>>>> +
>>>>> + if (!root)
>>>>> + root = root_mem_cgroup;
>>>>> +
>>>>> + for (; memcg; memcg = parent_mem_cgroup(memcg)) {
>>>>> + /* root_mem_cgroup does not have a soft limit */
>>>>> + if (memcg == root_mem_cgroup)
>>>>> + break;
>>>>> + if (res_counter_soft_limit_excess(&memcg->res))
>>>>> + return true;
>>>>> + if (memcg == root)
>>>>> + break;
>>>>> + }
>>>> Here it adds pressure on a cgroup if one of its parents exceeds soft
>>>> limit, although the cgroup itself is under soft limit. It does change
>>>> my understanding of soft limit, and might introduce regression of our
>>>> existing use cases.
>>>>
>>>> Here is an example:
>>>>
>>>> Machine capacity 32G and we over-commit by 8G.
>>>>
>>>> root
>>>> -> A (hard limit 20G, soft limit 15G, usage 16G)
>>>> -> A1 (soft limit 5G, usage 4G)
>>>> -> A2 (soft limit 10G, usage 12G)
>>>> -> B (hard limit 20G, soft limit 10G, usage 16G)
>>>>
>>>> under global reclaim, we don't want to add pressure on A1 although its
>>>> parent A exceeds its soft limit. Assume that if we set the soft limit
>>>> corresponding to each cgroup's working set size (hot memory), and it
>>>> will introduce regression to A1 in that case.
>>>>
>>>> In my existing implementation, i am checking the cgroup's soft limit
>>>> standalone w/o looking its ancestors.
>>> Why do you set the soft limit of A in the first place if you don't
>>> want it to be enforced?
>> The soft limit should be enforced under certain condition, not always.
>> The soft limit of A is set to be enforced when the parent of A and B
>> is under memory pressure. For example:
>>
>> Machine capacity 32G and we over-commit by 8G
>>
>> root
>> -> A (hard limit 20G, soft limit 12G, usage 20G)
>> -> A1 (soft limit 2G, usage 1G)
>> -> A2 (soft limit 10G, usage 19G)
>> -> B (hard limit 20G, soft limit 10G, usage 0G)
>>
>> Now, A is under memory pressure since the total usage is hitting its
>> hard limit. Then we start hierarchical reclaim under A, and each
>> cgroup under A also takes consideration of soft limit. In this case,
>> we should only set priority = 0 to A2 since it contributes to A's
>> charge as well as exceeding its own soft limit. Why punishing A1 (set
>> priority = 0) also which has usage under its soft limit ? I can
>> imagine it will introduce regression to existing environment which the
>> soft limit is set based on the working set size of the cgroup.
>>
>> To answer the question why we set soft limit to A, it is used to
>> over-commit the host while sharing the resource with its sibling (B in
>> this case). If the machine is under memory contention, we would like
>> to push down memory to A or B depends on their usage and soft limit.
> D'oh, I think the problem is just that we walk up the hierarchy one
> too many when checking whether a group exceeds a soft limit. The soft
> limit is a signal to distribute pressure that comes from above, it's
> meaningless and should indeed be ignored on the level the pressure
> originates from.
>
> Say mem_cgroup_over_soft_limit(root, memcg) would check everyone up to
> but not including root, wouldn't that do exactly what we both want?
>
> Example:
>
> 1. If global memory is short, we reclaim with root=root_mem_cgroup.
> A1 and A2 get soft limit reclaimed because of A's soft limit
> excess, just like the current kernel would do.
>
> 2. If A hits its hard limit, we reclaim with root=A, so we only mind
> the soft limits of A1 and A2. A1 is below its soft limit, all
> good. A2 is above its soft limit, gets treated accordingly. This
> is new behaviour, the current kernel would just reclaim them
> equally.
>
> Code:
>
> bool mem_cgroup_over_soft_limit(struct mem_cgroup *root,
> struct mem_cgroup *memcg)
> {
> if (mem_cgroup_disabled())
> return false;
>
> if (!root)
> root = root_mem_cgroup;
>
> for (; memcg; memcg = parent_mem_cgroup(memcg)) {
> if (memcg == root)
> break;
> if (res_counter_soft_limit_excess(&memcg->res))
> return true;
> }
> return false;
> }
Hi Johannes,
I don't think it solve the root of the problem, example:
root
-> A (hard limit 20G, soft limit 12G, usage 20G)
-> A1 ( soft limit 2G, usage 1G)
-> A2 ( soft limit 10G, usage 19G)
->B1 (soft limit 5G, usage 4G)
->B2 (soft limit 5G, usage 15G)
Now A is hitting its hard limit and start hierarchical reclaim under A.
If we choose B1 to go through mem_cgroup_over_soft_limit, it will
return true because its parent A2 has a large usage and will lead to
priority=0 reclaiming. But in fact it should be B2 to be punished.
IMHO, it may checking the cgroup's soft limit standalone without
looking up its ancestors just as Ying said.
Thanks,
Sha
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists