linux-kernel - Re: [PATCH v8 8/9] x86/virt/tdx: Reduce TDMR's reserved areas by using CMRs to find memory holes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <14ddb7fb218cd8ebcc9eca869a6f6b922212420b.camel@intel.com>
Date: Thu, 5 Dec 2024 12:45:51 +0000
From: "Huang, Kai" <kai.huang@...el.com>
To: "Hansen, Dave" <dave.hansen@...el.com>, "seanjc@...gle.com"
	<seanjc@...gle.com>, "bp@...en8.de" <bp@...en8.de>, "peterz@...radead.org"
	<peterz@...radead.org>, "hpa@...or.com" <hpa@...or.com>, "mingo@...hat.com"
	<mingo@...hat.com>, "tglx@...utronix.de" <tglx@...utronix.de>,
	"kirill.shutemov@...ux.intel.com" <kirill.shutemov@...ux.intel.com>,
	"pbonzini@...hat.com" <pbonzini@...hat.com>, "Williams, Dan J"
	<dan.j.williams@...el.com>
CC: "Yamahata, Isaku" <isaku.yamahata@...el.com>, "nik.borisov@...e.com"
	<nik.borisov@...e.com>, "Hunter, Adrian" <adrian.hunter@...el.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>, "Edgecombe,
 Rick P" <rick.p.edgecombe@...el.com>, "x86@...nel.org" <x86@...nel.org>
Subject: Re: [PATCH v8 8/9] x86/virt/tdx: Reduce TDMR's reserved areas by
 using CMRs to find memory holes

On Wed, 2024-12-04 at 14:22 +0000, Huang, Kai wrote:
> On Thu, 2024-11-14 at 00:57 +1300, Kai Huang wrote:
> > A TDX module initialization failure was reported on a Emerald Rapids
> > platform [*]:
> > 
> >   virt/tdx: initialization failed: TDMR [0x0, 0x80000000): reserved areas exhausted.
> >   virt/tdx: module initialization failed (-28)
> > 
> > As part of initializing the TDX module, the kernel informs the TDX
> > module of all "TDX-usable memory regions" using an array of TDX defined
> > structure "TD Memory Region" (TDMR).  Each TDMR must be in 1GB aligned
> > and in 1GB granularity, and all "non-TDX-usable memory holes" within a
> > given TDMR are marked as "reserved areas".  The TDX module reports a
> > maximum number of reserved areas that can be supported per TDMR (16).
> > 
> > The kernel builds the "TDX-usable memory regions" based on memblocks
> > (which reflects e820), and uses this list to find all "reserved areas"
> > for each TDMR.
> > 
> > It turns out that the kernel's view of memory holes is too fine grained
> > and sometimes exceeds the number of holes that the TDX module can track
> > per TDMR [1], resulting in the above failure.
> > 
> > Thankfully the module also lists memory that is potentially convertible
> > in a list of "Convertible Memory Regions" (CMRs).  That coarser grained
> > CMR list tends to track usable memory in the memory map even if it might
> > be reserved for host usage like 'ACPI data' [2].
> > 
> > Use that list to relax what the kernel considers unusable memory.  If it
> > falls in a CMR no need to instantiate a hole, and rely on the fact that
> > kernel will keep what it considers 'reserved' out of the page allocator.
> > 
> 
> Hi Dave,
> 
> I realized there's one issue if we change to use CMRs to fill up reserved areas
> for TDMRs after some internal discussion with Dan:
> 
> Currently we requires all memory regions in memblocks.memory to be TDX
> convertible memory at the time of initializing the TDX module to make module
> initialization successful.  We build a list of those memory regions as a list of
> "TDX memory blocks", and use them to construct the TDMRs to configure the
> module.  But we don't explicitly check those memory regions against CMRs to make
> sure they are truly TDX convertible, instead we depend on TDH.SYS.CONFIG to
> catch any non-CMR memory regions that end up to the TDX memory blocks.
> 
> This works fine because currently we use those TDX memory blocks to fill up the
> reserved areas of TDMRs, i.e., any non-CMR regions in TDX memory blocks will end
> up to "non-reserved areas" in TDMR(s), and this will cause TDH.SYS.CONFIG to
> fail.
> 
> After we change to using CMRs to fill up reserved areas of TDMRs, then all non-
> CMR regions will be marked as "reserved areas", regardless whether there is any
> non-CMR memory region in the TDX memory blocks.  This will result in TDX module
> initialization being successful while there are non-CMR pages going into page
> allocator.
> 
> To avoid this, we need to explicitly check all the TDX memory blocks against
> CMRs to make sure they are actually TDX convertible, before using CMRs to fill
> up reserved areas.
> 
> I ended up with below incremental diff with some additional text for the
> changelog.
> 
> Do you have any comments?
> 

Hi Dave,

I sent out the updated patch as "PATCH v8 8.1/9".  I appreciate if you can
review.

Hi Dan,

I didn't remove your Reviewed-by because I think the additional code added
should be straightforward.  But please let me know if you have any concern.

Thanks!