linux-kernel - Re: [RFC] memory reserve for userspace oom-killer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6217dcce-d178-3ab4-5174-1a98cdb14d98@sony.com>
Date:   Thu, 22 Apr 2021 15:41:05 +0000
From:   <Peter.Enderborg@...y.com>
To:     <shakeelb@...gle.com>
CC:     <hannes@...xchg.org>, <guro@...com>, <mhocko@...nel.org>,
        <linux-mm@...ck.org>, <akpm@...ux-foundation.org>,
        <cgroups@...r.kernel.org>, <rientjes@...gle.com>,
        <linux-kernel@...r.kernel.org>, <surenb@...gle.com>,
        <gthelen@...gle.com>, <dragoss@...gle.com>,
        <padmapriyad@...gle.com>
Subject: Re: [RFC] memory reserve for userspace oom-killer

On 4/22/21 4:27 PM, Shakeel Butt wrote:
> On Wed, Apr 21, 2021 at 10:39 PM <Peter.Enderborg@...y.com> wrote:
>> On 4/21/21 9:18 PM, Shakeel Butt wrote:
>>> On Wed, Apr 21, 2021 at 11:46 AM <Peter.Enderborg@...y.com> wrote:
>>>> On 4/21/21 8:28 PM, Shakeel Butt wrote:
>>>>> On Wed, Apr 21, 2021 at 10:06 AM peter enderborg
>>>>> <peter.enderborg@...y.com> wrote:
>>>>>> On 4/20/21 3:44 AM, Shakeel Butt wrote:
>>>>> [...]
>>>>>> I think this is the wrong way to go.
>>>>> Which one? Are you talking about the kernel one? We already talked out
>>>>> of that. To decide to OOM, we need to look at a very diverse set of
>>>>> metrics and it seems like that would be very hard to do flexibly
>>>>> inside the kernel.
>>>> You dont need to decide to oom, but when oom occurs you
>>>> can take a proper action.
>>> No, we want the flexibility to decide when to oom-kill. Kernel is very
>>> conservative in triggering the oom-kill.
>> It wont do it for you. We use this code to solve that:
> Sorry what do you mean by "It wont do it for you"?
The oom-killer, it does not do what you want and need.

You need to add something that kills the "right" task.
The example does that, it pick the task with highest
oom_score_adj and kills it. It is probably easer
to see in the "proof of concept" patch.

>
> [...]
>> int __init lowmemorykiller_register_oom_notifier(void)
>> {
>>     register_oom_notifier(&lowmemorykiller_oom_nb);
> This code is using oom_notify_list. That is only called when the
> kernel has already decided to go for the oom-kill. My point was the
> kernel is very conservative in deciding to trigger the oom-kill and
> the applications can suffer for long. We already have solutions for
> this issue in the form of userspace oom-killers (Android's lmkd and
> Facebook's oomd) which monitors a diverse set of metrics to early
> detect the application suffering and trigger SIGKILLs to release the
> memory pressure on the system.
>
> BTW with the userspace oom-killers, we would like to avoid the kernel
> oom-killer and memory.swap.high has been introduced in the kernel for
> that purpose.

This it is a lifeline. It will keep the lmkd/activity manger going. It is not
a replacement it is helper. It gives the freedom to tune other
parts with out worrying to much about oom. (Assuming that
userspace still can handle kills like the kernel lmk did)