linux-kernel - Re: [PATCH] x86/virt/tdx: Make TDX and kexec mutually exclusive at runtime

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <974274f1d3db2ee470fcef14407b5c3e516fbcb8.camel@intel.com>
Date: Thu, 17 Apr 2025 17:07:32 +0000
From: "Edgecombe, Rick P" <rick.p.edgecombe@...el.com>
To: "tglx@...utronix.de" <tglx@...utronix.de>, "peterz@...radead.org"
	<peterz@...radead.org>, "mingo@...hat.com" <mingo@...hat.com>, "Hansen, Dave"
	<dave.hansen@...el.com>, "Huang, Kai" <kai.huang@...el.com>, "bp@...en8.de"
	<bp@...en8.de>
CC: "ashish.kalra@....com" <ashish.kalra@....com>, "seanjc@...gle.com"
	<seanjc@...gle.com>, "x86@...nel.org" <x86@...nel.org>, "sagis@...gle.com"
	<sagis@...gle.com>, "hpa@...or.com" <hpa@...or.com>, "Chatre, Reinette"
	<reinette.chatre@...el.com>, "kirill.shutemov@...ux.intel.com"
	<kirill.shutemov@...ux.intel.com>, "Williams, Dan J"
	<dan.j.williams@...el.com>, "pbonzini@...hat.com" <pbonzini@...hat.com>,
	"thomas.lendacky@....com" <thomas.lendacky@....com>, "Yamahata, Isaku"
	<isaku.yamahata@...el.com>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "nik.borisov@...e.com" <nik.borisov@...e.com>
Subject: Re: [PATCH] x86/virt/tdx: Make TDX and kexec mutually exclusive at
 runtime

On Thu, 2025-04-17 at 11:02 +1200, Kai Huang wrote:
> Currently, kexec doesn't work well with TDX host support, and only one
> of them can be enabled in Kconfig.
> 

Nit:
"only one of them can be enabled in Kconfig at a time." would be a little
clearer.

>   However, distributions typically
> prefer to use a unified Kconfig with all features enabled.  Therefore,
> it would be very useful if both TDX host and kexec could be enabled in
> Kconfig simultaneously.
> 
> Full support for kexec on a TDX host would require complex work.  The
> cache flushing required would need to happen while stopping remote CPUs,
> which would require changes to a fragile area of the kernel.  It would
> also require resetting TDX private pages, which is non-trivial since the
> core kernel does not track them.  Lastly, it would have to rely on a
> yet-to-be documented behavior around the TME key (KeyID 0).
> 
> Leave the full support and the documentation clarification for future
> work, but start with a simple solution: change to make them mutually
> exclusive at runtime so that they can be both enabled in the Kconfig.
> 
> While there is a little bit of TDX setup at boot, the kexec sensitive
> parts of the initialization are enabled when KVM is loaded with a
> specific non-default kernel parameter (kvm_intel.tdx=Y).  Use a simple
> policy to decide which to run: whichever gets run first disables the
> other.  This effectively makes kexec race with TDX when KVM module is
> loaded.
> 
> Kexec has two phases: the kernel image loading phase and the actual
> execution phase.  Specifically, try to disable TDX permanently during
> the kernel image loading phase by leveraging the x86 version of
> machine_kexec_prepare().  If TDX has already been enabled (thus cannot
> be disabled), fail the kexec.
> 
> The lock that the TDX disabling operation takes is not held during the
> TDX per-CPU initialization, which happens before the main TDX
> initialization.  The consequence is that while kexec can't race with
> TDX initialization in a way that would leave private memory in a state
> that could corrupt the second kernel, it won't exclude the case of the
> TDX module being partially initialized.  In this rare scenario, TDX
> initialization would simply fail in the second kernel.  Keep the simple
> solution simple, and just document the race.
> 
> Another option could be to handle this when the kernel actually does
> kexec, but this would require adding an arch callout for the operation.
> Don't pursue this option to avoid complicating the kexec code.
> 
> If TDX cannot be disabled, the users will get an error:
> 
>   kexec_load failed: Operation not supported
> 
> This could be confusing to the users, thus also tell the reason in the
> dmesg:
> 
>   [..] kexec: Disabled: TDX is enabled
> 
> If TDX can be disabled, also print a message to let users know:
> 
>   [..] virt/tdx: explicitly disabled
> 
> The reason why this wasn't done earlier was the Kconfig option was just
> a bit simpler and the TDX code was large.  Moving to mutual exclusion at
> runtime is an incremental improvement that better meets the needs of
> distributions.
> 
> Signed-off-by: Kai Huang <kai.huang@...el.com>

I think this is good solution to start with. It has to be the simplest we could
do. Well, except just tweaking the Kconfig and documenting the race, but this
protection looks simple enough to be worth the cost. It would be good to get
distros on CC to confirm that this is acceptable.

Reviewed-by: Rick Edgecombe <rick.p.edgecombe@...el.com>