linux-kernel - Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page cache control

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1276613249.6437.11516.camel@nimitz>
Date:	Tue, 15 Jun 2010 07:47:29 -0700
From:	Dave Hansen <dave@...ux.vnet.ibm.com>
To:	Avi Kivity <avi@...hat.com>
Cc:	balbir@...ux.vnet.ibm.com, kvm <kvm@...r.kernel.org>,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [RFC/T/D][PATCH 2/2] Linux/Guest cooperative unmapped page
 cache control

On Tue, 2010-06-15 at 10:07 +0300, Avi Kivity wrote:
> On 06/14/2010 08:58 PM, Dave Hansen wrote:
> > On Mon, 2010-06-14 at 19:34 +0300, Avi Kivity wrote:
> >    
> >>> Again, this is useless when ballooning is being used.  But, I'm thinking
> >>> of a more general mechanism to force the system to both have MemFree
> >>> _and_ be acting as if it is under memory pressure.
> >>>
> >>>        
> >> If there is no memory pressure on the host, there is no reason for the
> >> guest to pretend it is under pressure.
> >>      
> > I can think of quite a few places where this would be beneficial.
> >
> > Ballooning is dangerous.  I've OOMed quite a few guests by
> > over-ballooning them.  Anything that's voluntary like this is safer than
> > things imposed by the host, although you do trade of effectiveness.
> 
> That's a bug that needs to be fixed.  Eventually the host will come 
> under pressure and will balloon the guest.  If that kills the guest, the 
> ballooning is not effective as a host memory management technique.

I'm not convinced that it's just a bug that can be fixed.  Consider a
case where a host sees a guest with 100MB of free memory at the exact
moment that a database app sees that memory.  The host tries to balloon
that memory away at the same time that the app goes and allocates it.
That can certainly lead to an OOM very quickly, even for very small
amounts of memory (much less than 100MB).  Where's the bug?

I think the issues are really fundamental to ballooning.

> > If all the guests do this, then it leaves that much more free memory on
> > the host, which can be used flexibly for extra host page cache, new
> > guests, etc...
> 
> If the host detects lots of pagecache misses it can balloon guests 
> down.  If pagecache is quiet, why change anything?

Page cache misses alone are not really sufficient.  This is the classic
problem where we try to differentiate streaming I/O (which we can't
effectively cache) from I/O which can be effectively cached.

> If the host wants to start new guests, it can balloon guests down.  If 
> no new guests are wanted, why change anything?

We're talking about an environment which we're always trying to
optimize.  Imagine that we're always trying to consolidate guests on to
smaller numbers of hosts.  We're effectively in a state where we
_always_ want new guests.

-- Dave

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/