linux-kernel - Re: [PATCH v5 3/7] x86/virt/tdx: Mark memory cache state incoherent when making SEAMCALL

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <f9631567afe3e72e7a99724092426cc06e887ed6.camel@intel.com>
Date: Mon, 4 Aug 2025 12:47:56 +0000
From: "Huang, Kai" <kai.huang@...el.com>
To: "Gao, Chao" <chao.gao@...el.com>
CC: "kvm@...r.kernel.org" <kvm@...r.kernel.org>, "ashish.kalra@....com"
	<ashish.kalra@....com>, "Hansen, Dave" <dave.hansen@...el.com>,
	"thomas.lendacky@....com" <thomas.lendacky@....com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"mingo@...hat.com" <mingo@...hat.com>, "dwmw@...zon.co.uk"
	<dwmw@...zon.co.uk>, "pbonzini@...hat.com" <pbonzini@...hat.com>,
	"seanjc@...gle.com" <seanjc@...gle.com>, "tglx@...utronix.de"
	<tglx@...utronix.de>, "kas@...nel.org" <kas@...nel.org>, "Chatre, Reinette"
	<reinette.chatre@...el.com>, "Yamahata, Isaku" <isaku.yamahata@...el.com>,
	"nik.borisov@...e.com" <nik.borisov@...e.com>, "hpa@...or.com"
	<hpa@...or.com>, "peterz@...radead.org" <peterz@...radead.org>,
	"sagis@...gle.com" <sagis@...gle.com>, "Chen, Farrah"
	<farrah.chen@...el.com>, "Edgecombe, Rick P" <rick.p.edgecombe@...el.com>,
	"bp@...en8.de" <bp@...en8.de>, "x86@...nel.org" <x86@...nel.org>, "Williams,
 Dan J" <dan.j.williams@...el.com>
Subject: Re: [PATCH v5 3/7] x86/virt/tdx: Mark memory cache state incoherent
 when making SEAMCALL

On Fri, 2025-08-01 at 16:23 +0800, Chao Gao wrote:
> On Tue, Jul 29, 2025 at 12:28:37AM +1200, Kai Huang wrote:
> > On TDX platforms, dirty cacheline aliases with and without encryption
> > bits can coexist, and the cpu can flush them back to memory in random
> > order.  During kexec, the caches must be flushed before jumping to the
> > new kernel otherwise the dirty cachelines could silently corrupt the
> > memory used by the new kernel due to different encryption property.
> > 
> > A percpu boolean is used to mark whether the cache of a given CPU may be
> > in an incoherent state, and the kexec performs WBINVD on the CPUs with
> > that boolean turned on.
> > 
> > For TDX, only the TDX module or the TDX guests can generate dirty
> > cachelines of TDX private memory, i.e., they are only generated when the
> > kernel does a SEAMCALL.
> > 
> > Set that boolean when the kernel does SEAMCALL so that kexec can flush
> > the cache correctly.
> > 
> > The kernel provides both the __seamcall*() assembly functions and the
> > seamcall*() wrapper ones which additionally handle running out of
> > entropy error in a loop.  Most of the SEAMCALLs are called using the
> > seamcall*(), except TDH.VP.ENTER and TDH.PHYMEM.PAGE.RDMD which are
> > called using __seamcall*() variant directly.
> > 
> > To cover the two special cases, add a new helper do_seamcall() which
> > only sets the percpu boolean and then calls the __seamcall*(), and
> > change the special cases to use do_seamcall().  To cover all other
> > SEAMCALLs, change seamcall*() to call do_seamcall().
> > 
> > For the SEAMCALLs invoked via seamcall*(), they can be made from both
> > task context and IRQ disabled context.  Given SEAMCALL is just a lengthy
> > instruction (e.g., thousands of cycles) from kernel's point of view and
> > preempt_{disable|enable}() is cheap compared to it, just unconditionally
> > disable preemption during setting the boolean and making SEAMCALL.
> > 
> > Signed-off-by: Kai Huang <kai.huang@...el.com>
> > Tested-by: Farrah Chen <farrah.chen@...el.com>
> 
> Reviewed-by: Chao Gao <chao.gao@...el.com>
> 
> One small question below,
> 
> <snip>
> 
> > +static __always_inline u64 do_seamcall(sc_func_t func, u64 fn,
> > +				       struct tdx_module_args *args)
> > +{
> > +	lockdep_assert_preemption_disabled();
> > +
> > +	/*
> > +	 * SEAMCALLs are made to the TDX module and can generate dirty
> > +	 * cachelines of TDX private memory.  Mark cache state incoherent
> > +	 * so that the cache can be flushed during kexec.
> > +	 *
> > +	 * This needs to be done before actually making the SEAMCALL,
> 
> ...
> 
> > +	 * because kexec-ing CPU could send NMI to stop remote CPUs,
> > +	 * in which case even disabling IRQ won't help here.
> > +	 */
> > +	this_cpu_write(cache_state_incoherent, true);
> 
> I'm not sure this write is guaranteed to occur before the SEAMCALL, as I don't
> see any explicit barrier to prevent the compiler from reordering them. Do you
> think this is an issue?

AFAICT this_cpu_write() eventually ended up with "volatile" being used, so
it has a compiler barrier already.

Plus what right after this_cpu_write() is a function call, with the body
actually implemented in another file, so I don't think the compiler is
smart enough to do any reordering here.

> 
> > +
> > +	return func(fn, args);
> > +}
> > +