Message-ID: <74311e891016f9c8480229aec171142b3b0bf4bb.camel@intel.com>
Date: Fri, 16 May 2025 17:04:47 +0000
From: "Edgecombe, Rick P" <rick.p.edgecombe@...el.com>
To: "seanjc@...gle.com" <seanjc@...gle.com>, "jgg@...pe.ca" <jgg@...pe.ca>
CC: "pvorel@...e.cz" <pvorel@...e.cz>, "kvm@...r.kernel.org"
	<kvm@...r.kernel.org>, "catalin.marinas@....com" <catalin.marinas@....com>,
	"Miao, Jun" <jun.miao@...el.com>, "Shutemov, Kirill"
	<kirill.shutemov@...el.com>, "pdurrant@...zon.co.uk" <pdurrant@...zon.co.uk>,
	"steven.price@....com" <steven.price@....com>, "peterx@...hat.com"
	<peterx@...hat.com>, "palmer@...belt.com" <palmer@...belt.com>,
	"amoorthy@...gle.com" <amoorthy@...gle.com>, "tabba@...gle.com"
	<tabba@...gle.com>, "quic_svaddagi@...cinc.com" <quic_svaddagi@...cinc.com>,
	"maz@...nel.org" <maz@...nel.org>, "vkuznets@...hat.com"
	<vkuznets@...hat.com>, "x86@...nel.org" <x86@...nel.org>, "keirf@...gle.com"
	<keirf@...gle.com>, "hughd@...gle.com" <hughd@...gle.com>, "Annapurve,
 Vishal" <vannapurve@...gle.com>, "mail@...iej.szmigiero.name"
	<mail@...iej.szmigiero.name>, "jack@...e.cz" <jack@...e.cz>, "Wieczor-Retman,
 Maciej" <maciej.wieczor-retman@...el.com>, "Zhao, Yan Y"
	<yan.y.zhao@...el.com>, "Du, Fan" <fan.du@...el.com>, "willy@...radead.org"
	<willy@...radead.org>, "paul.walmsley@...ive.com" <paul.walmsley@...ive.com>,
	"nsaenz@...zon.es" <nsaenz@...zon.es>, "aik@....com" <aik@....com>,
	"usama.arif@...edance.com" <usama.arif@...edance.com>,
	"quic_mnalajal@...cinc.com" <quic_mnalajal@...cinc.com>, "fvdl@...gle.com"
	<fvdl@...gle.com>, "rppt@...nel.org" <rppt@...nel.org>,
	"quic_cvanscha@...cinc.com" <quic_cvanscha@...cinc.com>, "Hansen, Dave"
	<dave.hansen@...el.com>, "vbabka@...e.cz" <vbabka@...e.cz>,
	"bfoster@...hat.com" <bfoster@...hat.com>, "quic_eberman@...cinc.com"
	<quic_eberman@...cinc.com>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "thomas.lendacky@....com"
	<thomas.lendacky@....com>, "mic@...ikod.net" <mic@...ikod.net>,
	"oliver.upton@...ux.dev" <oliver.upton@...ux.dev>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	"anthony.yznaga@...cle.com" <anthony.yznaga@...cle.com>,
	"binbin.wu@...ux.intel.com" <binbin.wu@...ux.intel.com>,
	"muchun.song@...ux.dev" <muchun.song@...ux.dev>, "Li, Zhiquan1"
	<zhiquan1.li@...el.com>, "rientjes@...gle.com" <rientjes@...gle.com>,
	"mpe@...erman.id.au" <mpe@...erman.id.au>, "Aktas, Erdem"
	<erdemaktas@...gle.com>, "david@...hat.com" <david@...hat.com>,
	"ajones@...tanamicro.com" <ajones@...tanamicro.com>, "jhubbard@...dia.com"
	<jhubbard@...dia.com>, "Xu, Haibo1" <haibo1.xu@...el.com>,
	"anup@...infault.org" <anup@...infault.org>, "Yamahata, Isaku"
	<isaku.yamahata@...el.com>, "jthoughton@...gle.com" <jthoughton@...gle.com>,
	"Wang, Wei W" <wei.w.wang@...el.com>, "steven.sistare@...cle.com"
	<steven.sistare@...cle.com>, "quic_pheragu@...cinc.com"
	<quic_pheragu@...cinc.com>, "jarkko@...nel.org" <jarkko@...nel.org>,
	"chenhuacai@...nel.org" <chenhuacai@...nel.org>, "Huang, Kai"
	<kai.huang@...el.com>, "shuah@...nel.org" <shuah@...nel.org>,
	"dwmw@...zon.co.uk" <dwmw@...zon.co.uk>, "pankaj.gupta@....com"
	<pankaj.gupta@....com>, "Peng, Chao P" <chao.p.peng@...el.com>,
	"nikunj@....com" <nikunj@....com>, "Graf, Alexander" <graf@...zon.com>,
	"viro@...iv.linux.org.uk" <viro@...iv.linux.org.uk>, "pbonzini@...hat.com"
	<pbonzini@...hat.com>, "yuzenghui@...wei.com" <yuzenghui@...wei.com>,
	"jroedel@...e.de" <jroedel@...e.de>, "suzuki.poulose@....com"
	<suzuki.poulose@....com>, "jgowans@...zon.com" <jgowans@...zon.com>, "Xu,
 Yilun" <yilun.xu@...el.com>, "liam.merwick@...cle.com"
	<liam.merwick@...cle.com>, "michael.roth@....com" <michael.roth@....com>,
	"quic_tsoni@...cinc.com" <quic_tsoni@...cinc.com>,
	"richard.weiyang@...il.com" <richard.weiyang@...il.com>, "Weiny, Ira"
	<ira.weiny@...el.com>, "aou@...s.berkeley.edu" <aou@...s.berkeley.edu>, "Li,
 Xiaoyao" <xiaoyao.li@...el.com>, "qperret@...gle.com" <qperret@...gle.com>,
	"kent.overstreet@...ux.dev" <kent.overstreet@...ux.dev>,
	"dmatlack@...gle.com" <dmatlack@...gle.com>, "james.morse@....com"
	<james.morse@....com>, "brauner@...nel.org" <brauner@...nel.org>,
	"ackerleytng@...gle.com" <ackerleytng@...gle.com>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"pgonda@...gle.com" <pgonda@...gle.com>, "quic_pderrin@...cinc.com"
	<quic_pderrin@...cinc.com>, "roypat@...zon.co.uk" <roypat@...zon.co.uk>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>, "will@...nel.org"
	<will@...nel.org>, "hch@...radead.org" <hch@...radead.org>
Subject: Re: [RFC PATCH v2 00/51] 1G page support for guest_memfd

On Fri, 2025-05-16 at 10:09 -0300, Jason Gunthorpe wrote:
> > You're conflating two different things.  guest_memfd allocating and managing
> > 1GiB physical pages, and KVM mapping memory into the guest at 1GiB/2MiB
> > granularity.  Allocating memory in 1GiB chunks is useful even if KVM can
> > only
> > map memory into the guest using 4KiB pages.
> 
> Even if KVM is limited to 4K the IOMMU might not be - alot of these
> workloads have a heavy IO component and we need the iommu to perform
> well too.

Oh, interesting point.

> 
> Frankly, I don't think there should be objection to making memory more
> contiguous. 

No objections from me to anything except the lack of concrete justification.

> There is alot of data that this always brings wins
> somewhere for someone.

For the direct map huge page benchmarking, they saw that sometimes 1GB pages
helped, but also sometimes 2MB pages helped. That 1GB will help *some* workload
doesn't seem surprising.

> 
> > The longer term goal of guest_memfd is to make it suitable for backing all
> > VMs,
> > hence Vishal's "Non-CoCo VMs" comment.  Yes, some of this is useful for TDX,
> > but
> > we (and others) want to use guest_memfd for far more than just CoCo VMs. 
> > And
> > for non-CoCo VMs, 1GiB hugepages are mandatory for various workloads.
> 
> Yes, even from an iommu perspective with 2D translation we need to
> have the 1G pages from the S2 resident in the IOTLB or performance
> falls off a cliff.

"falls off a cliff" is the level of detail and the direction of hand waving I
have been hearing. But it also seems modern CPUs are quite good at hiding the
cost of walks with caches etc. Like how 5 level paging was made unconditional. I
didn't think about IOTLB though. Thanks for mentioning it.

Powered by blists - more mailing lists