linux-kernel - Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 20 Dec 2017 12:15:46 -0800
From:   Shakeel Butt <shakeelb@...gle.com>
To:     Tejun Heo <tj@...nel.org>
Cc:     Michal Hocko <mhocko@...nel.org>, Li Zefan <lizefan@...wei.com>,
        Roman Gushchin <guro@...com>,
        Vladimir Davydov <vdavydov.dev@...il.com>,
        Greg Thelen <gthelen@...gle.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Hugh Dickins <hughd@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linux MM <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Cgroups <cgroups@...r.kernel.org>, linux-doc@...r.kernel.org
Subject: Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Wed, Dec 20, 2017 at 11:37 AM, Tejun Heo <tj@...nel.org> wrote:
> Hello, Shakeel.
>
> On Tue, Dec 19, 2017 at 02:39:19PM -0800, Shakeel Butt wrote:
>> Suppose a user wants to run multiple instances of a specific job on
>> different datacenters and s/he has budget of 100MiB for each instance.
>> The instances are schduled on the requested datacenters and the
>> scheduler has set the memory limit of those instances to 100MiB. Now,
>> some datacenters have swap deployed, so, there, let's say, the swap
>> limit of those instances are set according to swap medium
>> availability. In this setting the user will see inconsistent memcg OOM
>> behavior. Some of the instances see OOMs at 100MiB usage (suppose only
>> anon memory) while some will see OOMs way above 100MiB due to swap.
>> So, the user is required to know the internal knowledge of datacenters
>> (like which has swap or not and swap type) and has to set the limits
>> accordingly and thus increase the chance of config bugs.
>
> I don't understand how this invariant is useful across different
> backing swap devices and availability.  e.g. Our OOM decisions are
> currently not great in that the kernel can easily thrash for a very
> long time without making actual progresses.  If you combine that with
> widely varying types and availability of swaps,

The kernel never swaps out on hitting memsw limit. So, the varying
types and availability of swaps becomes invariant to the memcg OOM
behavior of the job.

> whether something is
> OOMing or not doesn't really tell you much.  The workload could be
> running completely fine or have been thrashing without making any
> meaningful forward progress for the past 15 mins.
>
> Given that whether or not swap exists, how much is avialable and how
> fast the backing swap device is all highly influential parameters in
> how the workload behaves, I don't see what having sum of memory + swap
> as an invariant actually buys.  And, even that essentially meaningless
> invariant doesn't really exist - the performance of the swap device
> absolutely affects when the OOM killer would kick in.
>

No, as I previously explained, the swap types and availability will be
transparent to the memcg OOM killer and memcg memory reclaim behavior.

> So, I don't see how the sum of memory+swap makes it possible to ignore
> the swap type and availability.  Can you please explain that further?
>
>> Also different types and sizes of swap mediums in data center will
>> further complicates the configuration. One datacenter might have SSD
>> as a swap, another might be doing swap on zram and third might be
>> doing swap on nvdimm. Each can have different size and can be assigned
>> to jobs differently. So, it is possible that the instances of the same
>> job might be assigned different swap limit on different datacenters.
>
> Sure, but what does memswap achieve?
>

1. memswap provides consistent memcg OOM killer and memcg memory
reclaim behavior independent to swap.
2. With memswap, the job owners do not have to think or worry about swaps.