linux-kernel - Re: [PATCH] staging, android: remove lowmemory killer from the tree

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 24 Feb 2017 14:16:34 +0100
From:   peter enderborg <peter.enderborg@...ymobile.com>
To:     Michal Hocko <mhocko@...nel.org>
CC:     Martijn Coenen <maco@...gle.com>,
        John Stultz <john.stultz@...aro.org>,
        Greg KH <gregkh@...uxfoundation.org>,
        Arve Hjønnevåg <arve@...roid.com>,
        Riley Andrews <riandrews@...roid.com>,
        <devel@...verdev.osuosl.org>, LKML <linux-kernel@...r.kernel.org>,
        Linux-MM <linux-mm@...ck.org>, Todd Kjos <tkjos@...gle.com>,
        Android Kernel Team <kernel-team@...roid.com>,
        Rom Lemarchand <romlem@...gle.com>,
        Tim Murray <timmurray@...gle.com>
Subject: Re: [PATCH] staging, android: remove lowmemory killer from the tree

On 02/24/2017 01:28 PM, Michal Hocko wrote:
> On Fri 24-02-17 13:19:46, peter enderborg wrote:
>> On 02/23/2017 09:36 PM, Martijn Coenen wrote:
>>> On Thu, Feb 23, 2017 at 9:24 PM, John Stultz <john.stultz@...aro.org> wrote:
>>>> So, just for context, Android does have a userland LMK daemon (using
>>>> the mempressure notifiers) as you mentioned, but unfortunately I'm
>>>> unaware of any devices that ship with that implementation.
>>> I've previously worked on enabling userspace lmkd for a previous
>>> release, but ran into some issues there (see below).
>>>
>>>> This is reportedly because while the mempressure notifiers provide a
>>>> the signal to userspace, the work the deamon then has to do to look up
>>>> per process memory usage, in order to figure out who is best to kill
>>>> at that point was too costly and resulted in poor device performance.
>>> In particular, mempressure requires memory cgroups to function, and we
>>> saw performance regressions due to the accounting done in mem cgroups.
>>> At the time we didn't have enough time left to solve this before the
>>> release, and we reverted back to kernel lmkd.
>>>
>>>> So for shipping Android devices, the LMK is still needed. However, its
>>>> not critical for basic android development, as the system will
>>>> function without it.
>>> It will function, but it most likely will perform horribly (as the
>>> page cache will be trashed to such a level that the system will be
>>> unusable).
>>>
>>>> Additionally I believe most vendors heavily
>>>> customize the LMK in their vendor tree, so the value of having it in
>>>> staging might be relatively low.
>>>>
>>>> It would be great however to get a discussion going here on what the
>>>> ulmkd needs from the kernel in order to efficiently determine who best
>>>> to kill, and how we might best implement that.
>>> The two main issues I think we need to address are:
>>> 1) Getting the right granularity of events from the kernel; I once
>>> tried to submit a patch upstream to address this:
>>> https://lkml.org/lkml/2016/2/24/582
>>> 2) Find out where exactly the memory cgroup overhead is coming from,
>>> and how to reduce it or work around it to acceptable levels for
>>> Android. This was also on 3.10, and maybe this has long been fixed or
>>> improved in more recent kernel versions.
>>>
>>> I don't have cycles to work on this now, but I'm happy to talk to
>>> whoever picks this up on the Android side.
>> I sent some patches that is different approach. It still uses shrinkers
>> but it has a kernel part that do the kill part better than the old one
>> but it does it the android way. The future for this is get it triggered
>> with other path's than slab shrinker. But we will not continue unless
>> we get google-android to be part of it. Hocko objected heavy on
>> the patches but seems not to see that we need something to
>> do the job before we can disconnect from shrinker.
> Yeah, I strongly believe that the chosen approach is completely wrong.
> Both in abusing the shrinker interface and abusing oom_score_adj as the
> only criterion for the oom victim selection.

No one is arguing that shrinker is not problematic. And would be great if it is removed from lmk.
The oom_score_adj is the way user-space tells the kernel what the user-space has as prio. And android
is using that very much. It's a core part. I have never seen it be used on other
linux system so what is the intended usage of oom_score_adj? Is this really abusing?

I think I can help out with removing  shrinker from lmk. Not using oom_score_adj is harder and
has a bigger impact on android, except the trivial solution by adding replacement
oom_user_prio and use that within android and kernel.