linux-kernel - Re: [PATCH v6 7/7] KVM: TDX: Explicitly do WBINVD when no more TDX SEAMCALLs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <d2e33db367b503dde2f342de3cedb3b8fa29cc42.camel@intel.com>
Date: Thu, 14 Aug 2025 22:19:10 +0000
From: "Huang, Kai" <kai.huang@...el.com>
To: "seanjc@...gle.com" <seanjc@...gle.com>, "Edgecombe, Rick P"
	<rick.p.edgecombe@...el.com>
CC: "kvm@...r.kernel.org" <kvm@...r.kernel.org>, "ashish.kalra@....com"
	<ashish.kalra@....com>, "Hansen, Dave" <dave.hansen@...el.com>,
	"thomas.lendacky@....com" <thomas.lendacky@....com>, "kas@...nel.org"
	<kas@...nel.org>, "mingo@...hat.com" <mingo@...hat.com>, "dwmw@...zon.co.uk"
	<dwmw@...zon.co.uk>, "pbonzini@...hat.com" <pbonzini@...hat.com>, "Chatre,
 Reinette" <reinette.chatre@...el.com>, "Yamahata, Isaku"
	<isaku.yamahata@...el.com>, "nik.borisov@...e.com" <nik.borisov@...e.com>,
	"tglx@...utronix.de" <tglx@...utronix.de>, "hpa@...or.com" <hpa@...or.com>,
	"peterz@...radead.org" <peterz@...radead.org>, "sagis@...gle.com"
	<sagis@...gle.com>, "Chen, Farrah" <farrah.chen@...el.com>, "bp@...en8.de"
	<bp@...en8.de>, "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>, "binbin.wu@...ux.intel.com"
	<binbin.wu@...ux.intel.com>, "Gao, Chao" <chao.gao@...el.com>, "Williams, Dan
 J" <dan.j.williams@...el.com>, "x86@...nel.org" <x86@...nel.org>
Subject: Re: [PATCH v6 7/7] KVM: TDX: Explicitly do WBINVD when no more TDX
 SEAMCALLs

On Thu, 2025-08-14 at 11:00 -0700, Sean Christopherson wrote:
> On Thu, Aug 14, 2025, Rick P Edgecombe wrote:
> > On Thu, 2025-08-14 at 06:54 -0700, Sean Christopherson wrote:
> > > > diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> > > > index 66744f5768c8..1bc6f52e0cd7 100644
> > > > --- a/arch/x86/kvm/vmx/tdx.c
> > > > +++ b/arch/x86/kvm/vmx/tdx.c
> > > > @@ -442,6 +442,18 @@ void tdx_disable_virtualization_cpu(void)
> > > >   		tdx_flush_vp(&arg);
> > > >   	}
> > > >   	local_irq_restore(flags);
> > > > +
> > > > +	/*
> > > > +	 * No more TDX activity on this CPU from here.  Flush cache to
> > > > +	 * avoid having to do WBINVD in stop_this_cpu() during kexec.
> > > > +	 *
> > > > +	 * Kexec calls native_stop_other_cpus() to stop remote CPUs
> > > > +	 * before booting to new kernel, but that code has a "race"
> > > > +	 * when the normal REBOOT IPI times out and NMIs are sent to
> > > > +	 * remote CPUs to stop them.  Doing WBINVD in stop_this_cpu()
> > > > +	 * could potentially increase the possibility of the "race".
> 
> Why is that race problematic?  The changelog just says
> 
>  : However, the native_stop_other_cpus() and stop_this_cpu() have a "race"
>  : which is extremely rare to happen but could cause the system to hang.
>  : even
>  : Specifically, the native_stop_other_cpus() firstly sends normal reboot
>  : IPI to remote CPUs and waits one second for them to stop.  If that times
>  : out, native_stop_other_cpus() then sends NMIs to remote CPUs to stop
>  : them.
> 
> without explaining how that can cause a system hang.

Thanks for review. Sean.

The race is about the kexec-ing CPU could jump to second kernel when other
CPUs have not fully stopped.  

In the patch 3 I appended a link in the changelog to explain the race:

https://lore.kernel.org/kvm/b963fcd60abe26c7ec5dc20b42f1a2ebbcc72397.1750934177.git.kai.huang@intel.com/

Please see "[*] The "race" in native_stop_other_cpus()" part.

I will put the link in the changelog of this patch too.

> 
> > > > +	 */
> > > > +	tdx_cpu_flush_cache();
> > > 
> > > IIUC, this can be:
> > > 
> > > 	if (IS_ENABLED(CONFIG_KEXEC))
> > > 		tdx_cpu_flush_cache();
> > > 
> > 
> > No strong objection, just 2 cents. I bet !CONFIG_KEXEC && CONFIG_INTEL_TDX_HOST
> > kernels will be the minority. Seems like an opportunity to simplify the code.
> 
> Reducing the number of lines of code is not always a simplification.  IMO, not
> checking CONFIG_KEXEC adds "complexity" because anyone that reads the comment
> (and/or the massive changelog) will be left wondering why there's a bunch of
> documentation that talks about kexec, but no hint of kexec considerations in the
> code.

I think we can use 'kexec_in_progress', which is even better than
IS_ENABLED(CONFIG_KEXEC) IMHO.

When CONFIG_KEXEC is on, 'kexec_in_progress' will only be set when kexec
is actually happening, thus tdx_cpu_flush_cache() will only be called for
kexec.  When CONFIG_KEXEC (CONFIG_KEXEC_CORE) is off, then
'kexec_in_progress' is a macro defined to false.  The compiler can
optimize this out too I suppose.

Any comments?