linux-kernel - Re: [PATCH -v2 -mm] add extra free kbytes tunable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20111012122018.690bdf28.akpm@linux-foundation.org>
Date:	Wed, 12 Oct 2011 12:20:18 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Rik van Riel <riel@...hat.com>
Cc:	Satoru Moriya <satoru.moriya@....com>,
	David Rientjes <rientjes@...gle.com>,
	Randy Dunlap <rdunlap@...otime.net>,
	Satoru Moriya <smoriya@...hat.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"lwoodman@...hat.com" <lwoodman@...hat.com>,
	Seiji Aguchi <saguchi@...hat.com>,
	"hughd@...gle.com" <hughd@...gle.com>,
	"hannes@...xchg.org" <hannes@...xchg.org>
Subject: Re: [PATCH -v2 -mm] add extra free kbytes tunable

On Wed, 12 Oct 2011 09:09:17 -0400
Rik van Riel <riel@...hat.com> wrote:

> On 10/11/2011 04:54 PM, Andrew Morton wrote:
> > On Tue, 11 Oct 2011 16:23:22 -0400
> > Satoru Moriya<satoru.moriya@....com>  wrote:
> >
> >> On 10/11/2011 03:55 PM, Andrew Morton wrote:
> >>> On Tue, 11 Oct 2011 15:32:11 -0400
> >>> Satoru Moriya<satoru.moriya@....com>  wrote:
> >>>
> >>>> On 10/10/2011 06:37 PM, Andrew Morton wrote:
> >>>>> On Fri, 7 Oct 2011 20:08:19 -0700 (PDT) David Rientjes
> >>>>> <rientjes@...gle.com>  wrote:
> >>>>>
> >>>>>> On Thu, 1 Sep 2011, Rik van Riel wrote:
> >>>>
> >>>> Actually page allocator decreases min watermark to 3/4 * min
> >>>> watermark for rt-task. But in our case some applications create a lot
> >>>> of processes and if all of them are rt-task, the amount of watermark
> >>>> bonus(1/4 * min watermark) is not enough.
> >>>>
> >>>> If we can tune the amount of bonus, it may be fine. But that is
> >>>> almost all same as extra free kbytes.
> >>>
> >>> This situation is detectable at runtime.  If realtime tasks are being
> >>> stalled in the page allocator then start to increase the free-page
> >>> reserves.  A little control system.
> >>
> >> Detecting at runtime is too late for some latency critical systems.
> >> At that system, we must avoid a stall before it happens.
> >
> > It's pretty darn obvious that the kernel can easily see the situation
> > developing before it happens.  By comparing a few integers.
> 
> The problem is that we may be dealing with bursts, not steady
> states of allocations.  Without knowing the size of a burst,
> we have no idea when we should wake up kswapd to get enough
> memory freed ahead of the application's allocations.

That problem remains with this patch - it just takes a larger burst.

Unless the admin somehow manages to configure the tunable large enough
to cover the largest burst, and there aren't other applications
allocating memory during that burst, and the time between bursts is
sufficient for kswapd to be able to sufficiently replenish free-page
reserves.  All of which sounds rather unlikely.

> > Look, please don't go bending over backwards like this to defend a bad
> > patch.  It's a bad patch!  It would be better not to have to merge it.
> > Let's do something better.
> 
> I would love it if we could come up with something better,
> and have thought about it a lot.
> 
> However, so far we do not seem to have an alternative yet :(

Do we actually have a real-world application which is hurting from
this?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/