linux-kernel - Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAAMvbhH=ftMoh9eFVR3YgN9DVSLaN5tXa-vTsBocY8YuL0Rc1A@mail.gmail.com>
Date:   Wed, 21 Aug 2019 22:42:29 +0100
From:   James Courtier-Dutton <james.dutton@...il.com>
To:     Daniel Drake <drake@...lessm.com>
Cc:     "Artem S. Tashkinov" <aros@....com>,
        LKML Mailing List <linux-kernel@...r.kernel.org>,
        linux@...lessm.com, hadess@...ess.net,
        Johannes Weiner <hannes@...xchg.org>
Subject: Re: Let's talk about the elephant in the room - the Linux kernel's
 inability to gracefully handle low memory pressure

On Tue, 20 Aug 2019 at 07:47, Daniel Drake <drake@...lessm.com> wrote:
>
> Hi,
>
> And if there is a meaningful way to make the kernel behave better, that would
> obviously be of huge value too.
>
> Thanks
> Daniel

Hi,

Is there a way for an application to be told that there is a memory
pressure situation?
For example, say I do a "make -j32"  and half way through the compile
it hits a memory pressure situation.
If make could have been told about it. It could give up on some of the
parallel compiles, and instead proceed as if the user have typed "make
-j4". It could then re-try the failed compile parts, that failed due
to memory pressure.
I know all applications won't be this clever, but providing a kernel
API so that an application could do something about it, if the
programmer of that application has thought about it.
It could be similar with say, a hadoop application. If the hadoop
process detects memory pressure, if could back off, and process the
data more slowly and not try to do so much at the same time.
The kernel could also detect which processes are contributing most to
the memory pressure (trying to do mallocs) and give them less
processor time, and instead ask all processes to release some memory,
and those processes that understood the kernel API for that
notification could actually do something about it.

I have also found, for the desktop, one of the biggest pressures on
the system is disk pressure. Accessing the disk causes the UI to be
less responsive.
For example, if I am in vim, and trying to type letters on the
keyboard, whether some other application is using the disk or not
should have no impact on my letter writing. Has anyone got any ideas
with regards to what we can do about that?

Kind Regards

James