linux-kernel - Re: [PATCH] lowmemorykiller: Avoid excessive/redundant calling of LMK

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANcMJZALAz1WKjo+8VbUMWBpS117gaZht-b7jBLJWT9VVN83=g@mail.gmail.com>
Date:	Thu, 29 Jan 2015 16:44:51 -0800
From:	John Stultz <john.stultz@...aro.org>
To:	Michal Hocko <mhocko@...e.cz>
Cc:	Chintan Pandya <cpandya@...eaurora.org>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	Weijie Yang <weijie.yang@...sung.com>,
	David Rientjes <rientjes@...gle.com>,
	devel@...verdev.osuosl.org,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linux-MM <linux-mm@...ck.org>,
	Android Kernel Team <kernel-team@...roid.com>,
	Rom Lemarchand <romlem@...gle.com>,
	Anton Vorontsov <anton@...msg.org>
Subject: Re: [PATCH] lowmemorykiller: Avoid excessive/redundant calling of LMK

On Thu, Jan 15, 2015 at 9:03 AM, Michal Hocko <mhocko@...e.cz> wrote:
> On Mon 12-01-15 21:49:14, Chintan Pandya wrote:
>> The global shrinker will invoke lowmem_shrink in a loop.
>> The loop will be run (total_scan_pages/batch_size) times.
>> The default batch_size will be 128 which will make
>> shrinker invoking 100s of times. LMK does meaningful
>> work only during first 2-3 times and then rest of the
>> invocations are just CPU cycle waste. Fix that by returning
>> to the shrinker with SHRINK_STOP when LMK doesn't find any
>> more work to do. The deciding factor here is, no process
>> found in the selected LMK bucket or memory conditions are
>> sane.
>
> lowmemory killer is broken by design and this one of the examples which
> shows why. It simply doesn't fit into shrinkers concept.
>
> The count_object callback simply lies and tells the core that all
> the reclaimable LRU pages are scanable and gives it this as a number
> which the core uses for total_scan. scan_objects callback then happily
> ignore nr_to_reclaim and does its one time job where it iterates over
> _all_ tasks and picks up the victim and returns its rss as a return
> value. This is just a subset of LRU pages of course so it continues
> looping until total_scan goes down to 0 finally.
>
> If this really has to be a shrinker then, shouldn't it evaluate the OOM
> situation in the count callback and return non zero only if OOM and then
> the scan callback would kill and return nr_to_reclaim.
>
> Or even better wouldn't it be much better to use vmpressure to wake
> up a kernel module which would simply check the situation and kill
> something?
>
> Please do not put only cosmetic changes on top of broken concept and try
> to think about a proper solution that is what staging is for AFAIU.
>
> The code is in this state for quite some time and I would really hate if
> it got merged just because it is in staging for too long and it is used
> out there.

So the in-kernel low-memory-killer is hopefully on its way out.

With Lollipop on some devices, Android is using the mempressure
notifiers to kill processes from userland. However, not all devices
have moved to this new model (and possibly some resulting performance
issues are being worked out? Its not clear).  So hopefully we can drop
it soon, but I'd like to make sure we don't get only a half-working
solution upstream before we do remove it.

thanks
-john
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/