linux-kernel - RE: Frontswap [PATCH 0/4] (was Transcendent Memory): overview

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <d3bbdffe-8cc7-4ab9-8292-1531d13fad8e@default>
Date:	Mon, 3 May 2010 07:59:57 -0700 (PDT)
From:	Dan Magenheimer <dan.magenheimer@...cle.com>
To:	Avi Kivity <avi@...hat.com>
Cc:	ngupta@...are.org, Jeremy Fitzhardinge <jeremy@...p.org>,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	Pavel Machek <pavel@....cz>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, hugh.dickins@...cali.co.uk,
	JBeulich@...ell.com, chris.mason@...cle.com,
	kurt.hackel@...cle.com, dave.mccracken@...cle.com, npiggin@...e.de,
	akpm@...ux-foundation.org, riel@...hat.com
Subject: RE: Frontswap [PATCH 0/4] (was Transcendent Memory): overview

> > My analogy only requires some
> > statistical bad luck: Multiple guests with peaks and valleys
> > of memory requirements happen to have their peaks align.
> 
> Not sure I understand.

Virtualization is all about statistical multiplexing of fixed
resources.  If all guests demand a resource simultaneously,
that is peak alignment == "bad luck".

(But, honestly, I don't even remember the point either of us
was trying to make here :-)

> > Or maybe not... when a guest is in the middle of a live migration,
> > I believe (in Xen), the entire guest memory allocation (possibly
> > excluding ballooned-out pages) must be simultaneously in RAM briefly
> > in BOTH the host and target machine.  That is, live migration is
> > not "pipelined".  Is this also true of KVM?
> 
> No.  The entire guest address space can be swapped out on the source
> and
> target, less the pages being copied to or from the wire, and pages
> actively accessed by the guest.  Of course performance will suck if all
> memory is swapped out.

Will it suck to the point of eventually causing the live migration
to fail?  Or will swap-storms effectively cause denial-of-service
for other guests?

Anyway, if live migration works fine with mostly-swapped-out guests
on KVM, that's great.

> > Choosing the _optimal_ overcommit ratio is impossible without a
> > prescient knowledge of the workload in each guest.  Hoping memory
> > will be available is certainly not a good solution, but if memory
> > is not available guest swapping is much better than host swapping.
> 
> You cannot rely on guest swapping.

Frontswap only relies on the guest having an existing swap device,
defined in /etc/fstab like any normal Linux swap device.  If this
is "relying on guest swapping", yes frontswap relies on guest swapping.

Or if you are referring to your "host can't force guest to
reclaim pages" argument, see the other thread.

> > And making RAM usage as dynamic as possible and live migration
> > as easy as possible are keys to maximizing the benefits (and
> > limiting the problems) of virtualization.
> 
> That is why you need overcommit.  You make things dynamic with page
> sharing and ballooning and live migration, but at some point you need a
> failsafe fallback.  The only failsafe fallback I can see (where the
> host doesn't rely on guests) is swapping.

No fallback is required if the overcommitment is done intelligently.

> As far as I can tell, frontswap+tmem increases the problem.  You loan
> the guest some memory without the means to take it back, this increases
> memory pressure on the host.  The result is that if you want to avoid
> swapping (or are unable to) you need to undercommit host resources.
> Instead of sum(guest mem) + reserve < (host mem), you need sum(guest
> mem
> + committed tmem) + reserve < (host mem).  You need more host memory,
> or less guests, or to be prepared to swap if the worst happens.

Your argument might make sense from a KVM perspective but is
not true of frontswap with Xen+tmem.  With KVM, the host's
swap disk(s) can all be used as "slow RAM".  With Xen, there is
no host swap disk.  So, yes, the degree of potential memory
overcommitment is smaller with Xen+tmem than with KVM.  In
order to avoid all the host problems with host-swapping,
frontswap+Xen+tmem intentionally limits the degree of memory
overcommitment... but this is just memory overcommitment done
intelligently.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/