linux-kernel - Re: [PATCH] lowmemorykiller: Avoid excessive/redundant calling of LMK

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAABpnA-hGh2iP866aB+U7y6SN4pU2izP1wPUCYpkc+F7TQcDvw@mail.gmail.com>
Date:	Thu, 29 Jan 2015 18:08:58 -0800
From:	Rom Lemarchand <romlem@...gle.com>
To:	John Stultz <john.stultz@...aro.org>
Cc:	Michal Hocko <mhocko@...e.cz>,
	Chintan Pandya <cpandya@...eaurora.org>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	Weijie Yang <weijie.yang@...sung.com>,
	David Rientjes <rientjes@...gle.com>,
	"devel@...verdev.osuosl.org" <devel@...verdev.osuosl.org>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linux-MM <linux-mm@...ck.org>,
	Android Kernel Team <kernel-team@...roid.com>,
	Anton Vorontsov <anton@...msg.org>
Subject: Re: [PATCH] lowmemorykiller: Avoid excessive/redundant calling of LMK

On Thu, Jan 29, 2015 at 4:44 PM, John Stultz <john.stultz@...aro.org> wrote:
> On Thu, Jan 15, 2015 at 9:03 AM, Michal Hocko <mhocko@...e.cz> wrote:
>> On Mon 12-01-15 21:49:14, Chintan Pandya wrote:
>>> The global shrinker will invoke lowmem_shrink in a loop.
>>> The loop will be run (total_scan_pages/batch_size) times.
>>> The default batch_size will be 128 which will make
>>> shrinker invoking 100s of times. LMK does meaningful
>>> work only during first 2-3 times and then rest of the
>>> invocations are just CPU cycle waste. Fix that by returning
>>> to the shrinker with SHRINK_STOP when LMK doesn't find any
>>> more work to do. The deciding factor here is, no process
>>> found in the selected LMK bucket or memory conditions are
>>> sane.
>>
>> lowmemory killer is broken by design and this one of the examples which
>> shows why. It simply doesn't fit into shrinkers concept.
>>
>> The count_object callback simply lies and tells the core that all
>> the reclaimable LRU pages are scanable and gives it this as a number
>> which the core uses for total_scan. scan_objects callback then happily
>> ignore nr_to_reclaim and does its one time job where it iterates over
>> _all_ tasks and picks up the victim and returns its rss as a return
>> value. This is just a subset of LRU pages of course so it continues
>> looping until total_scan goes down to 0 finally.
>>
>> If this really has to be a shrinker then, shouldn't it evaluate the OOM
>> situation in the count callback and return non zero only if OOM and then
>> the scan callback would kill and return nr_to_reclaim.
>>
>> Or even better wouldn't it be much better to use vmpressure to wake
>> up a kernel module which would simply check the situation and kill
>> something?
>>
>> Please do not put only cosmetic changes on top of broken concept and try
>> to think about a proper solution that is what staging is for AFAIU.
>>
>> The code is in this state for quite some time and I would really hate if
>> it got merged just because it is in staging for too long and it is used
>> out there.
>
> So the in-kernel low-memory-killer is hopefully on its way out.
>
> With Lollipop on some devices, Android is using the mempressure
> notifiers to kill processes from userland. However, not all devices
> have moved to this new model (and possibly some resulting performance
> issues are being worked out? Its not clear).  So hopefully we can drop
> it soon, but I'd like to make sure we don't get only a half-working
> solution upstream before we do remove it.
>
> thanks
> -john

We are still working on a user space replacement to LMK. We have
definitely had issues with LMKd and so stayed with the in kernel one
for all the lollipop devices we shipped. Issues were mostly related to
performance, timing of OOM notifications and when under intense memory
pressure we ran into issues where even opening a file would fail due
to no RAM being available.
As John said, it's WIP and hopefully we'll be able to drop the in
kernel one soon.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/