Message-ID: <3f9064160b43df488d73302b3d736e23a9cd2b66.camel@amazon.com>
Date: Mon, 5 Aug 2024 19:47:56 +0000
From: "Gowans, James" <jgowans@...zon.com>
To: "pbonzini@...hat.com" <pbonzini@...hat.com>, "tytso@....edu"
	<tytso@....edu>
CC: "kvm@...r.kernel.org" <kvm@...r.kernel.org>, "rppt@...nel.org"
	<rppt@...nel.org>, "brauner@...nel.org" <brauner@...nel.org>, "Graf (AWS),
 Alexander" <graf@...zon.de>, "anthony.yznaga@...cle.com"
	<anthony.yznaga@...cle.com>, "steven.sistare@...cle.com"
	<steven.sistare@...cle.com>, "akpm@...ux-foundation.org"
	<akpm@...ux-foundation.org>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "seanjc@...gle.com" <seanjc@...gle.com>,
	"Woodhouse, David" <dwmw@...zon.co.uk>, "linux-mm@...ck.org"
	<linux-mm@...ck.org>, "nh-open-source@...zon.com"
	<nh-open-source@...zon.com>, "Saenz Julienne, Nicolas" <nsaenz@...zon.es>,
	"Durrant, Paul" <pdurrant@...zon.co.uk>, "viro@...iv.linux.org.uk"
	<viro@...iv.linux.org.uk>, "jack@...e.cz" <jack@...e.cz>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"jgg@...pe.ca" <jgg@...pe.ca>, "usama.arif@...edance.com"
	<usama.arif@...edance.com>
Subject: Re: [PATCH 00/10] Introduce guestmemfs: persistent in-memory filesystem

On Mon, 2024-08-05 at 16:41 +0200, Paolo Bonzini wrote:
> On Mon, Aug 5, 2024 at 4:35 PM Theodore Ts'o <tytso@....edu> wrote:
> > On Mon, Aug 05, 2024 at 11:32:35AM +0200, James Gowans wrote:
> > > Guestmemfs implements preservation acrosss kexec by carving out a
> > > large contiguous block of host system RAM early in boot which is
> > > then used as the data for the guestmemfs files.
> > 
> > Also, the VMM update process is not a common case thing, so we don't
> > need to optimize for performance.  If we need to temporarily use
> > swap/zswap to allocate memory at VMM update time, and if the pages
> > aren't contiguous when they are copied out before doing the VMM
> > update
> 
> I'm not sure I understand, where would this temporary allocation happen?

The intended use case for live update is to update the entirely of the
hypervisor: kexecing into a new kernel, launching new VMM processes. So
anything in kernel state (page tables, VMAs, (z)swap entries, struct
pages, etc) is all lost after kexec and needs to be re-created. That's
the job of guestmemfs: provide the persistence across kexec and ability
to re-create the mapping by re-opening the files.

It would be far too impactful to need to write out the whole VM memory
to disk. Also with CoCo VMs that's not really possible. When virtual
machines are running, every millisecond of down time counts. It would be
wasteful to need to keep terabytes of SSDs lying around just to briefly
write all the guest RAM there and then read it out a moment later. Much
better to leave all the guest memory where it is: in memory.

> 
> > that might be very well worth the vast of of memory needed to
> > pay for reserving memory on the host for the VMM update that only
> > might happen once every few days/weeks/months (depending on whether
> > you are doing update just for high severity security fixes, or for
> > random VMM updates).
> > 
> > Even if you are updating the VMM every few days, it still doesn't seem
> > that permanently reserving contiguous memory on the host can be
> > justified from a TCO perspective.
> 
> As far as I understand, this is intended for use in systems that do
> not do anything except hosting VMs, where anyway you'd devote 90%+ of
> host memory to hugetlbfs gigapages.

Exactly, the use case here is for machines whose only job is to be a KVM
hypervisor. The majority of system RAM is donated to guestmemfs;
anything else (host kernel memory and VMM anonymous memory) is
essentially overhead and should be minimised.

JG

Powered by blists - more mailing lists