linux-kernel - Re: [PATCH 0/3] Unmapped page cache control (v5)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sun,  3 Apr 2011 18:32:16 +0900 (JST)
From:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
To:	Dave Chinner <david@...morbit.com>
Cc:	kosaki.motohiro@...fujitsu.com, Christoph Lameter <cl@...ux.com>,
	Balbir Singh <balbir@...ux.vnet.ibm.com>, linux-mm@...ck.org,
	akpm@...ux-foundation.org, npiggin@...nel.dk, kvm@...r.kernel.org,
	linux-kernel@...r.kernel.org, kamezawa.hiroyu@...fujitsu.com,
	Mel Gorman <mel@....ul.ie>, Minchan Kim <minchan.kim@...il.com>
Subject: Re: [PATCH 0/3] Unmapped page cache control (v5)

> On Fri, Apr 01, 2011 at 10:17:56PM +0900, KOSAKI Motohiro wrote:
> > > > But, I agree that now we have to concern slightly large VM change parhaps
> > > > (or parhaps not). Ok, it's good opportunity to fill out some thing.
> > > > Historically, Linux MM has "free memory are waste memory" policy, and It
> > > > worked completely fine. But now we have a few exceptions.
> > > >
> > > > 1) RT, embedded and finance systems. They really hope to avoid reclaim
> > > >    latency (ie avoid foreground reclaim completely) and they can accept
> > > >    to make slightly much free pages before memory shortage.
> > > 
> > > In general we need a mechanism to ensure we can avoid reclaim during
> > > critical sections of application. So some way to give some hints to the
> > > machine to free up lots of memory (/proc/sys/vm/dropcaches is far too
> > > drastic) may be useful.
> > 
> > Exactly.
> > I've heard multiple times this request from finance people. And I've also 
> > heared the same request from bullet train control software people recently.
> 
> Well, that's enough to make me avoid Japanese trains in future. 

Feel free do. :)

>If
> your critical control system has problems with memory reclaim
> interfering with it's operation, then you are doing something
> very, very wrong.
> 
> If you have a need to avoid memory allocation latency during
> specific critical sections then the critical section needs to:
> 
> 	a) have all it's memory preallocated and mlock()d in advance
> 
> 	b) avoid doing anything that requires memory to be
> 	   allocated.
> 
> These are basic design rules for time-sensitive applications.

I wonder why do you think our VM folks don't know that.


> Fundamentally, if you just switch off memory reclaim to avoid the
> latencies involved with direct memory reclaim, then all you'll get
> instead is ENOMEM because there's no memory available and none will be
> reclaimed. That's even more fatal for the system than doing reclaim.

You have two level oversight.

Firstly, *ALL* RT application need to cooperate applications, kernel, 
and other various system level daemons. That's no specific issue of 
this topic. OK, *IF* RT application run egoistic, a system may hang 
up easily even routh mere simple busy loop, yes. But, Who want to do so?

Secondly, You misparsed "avoid direct reclaim" paragraph. We don't talk
about "avoid direct reclaim even if system memory is no enough", We talk
about "avoid direct reclaim by preparing before". 


> IMO, you should tell the people requesting stuff like this to
> architect their critical sections according to best practices.
> Hacking the VM to try to work around badly designed applications is
> a sure recipe for disaster...

I hope this mail satisfy you. :)


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/