linux-kernel - RE: [PATCH v9 07/18] x86/virt/tdx: Do TDX module per-cpu initialization

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <BL1PR11MB5978F562548EA22BFFD13970F7DD9@BL1PR11MB5978.namprd11.prod.outlook.com>
Date:   Mon, 13 Feb 2023 21:13:51 +0000
From:   "Huang, Kai" <kai.huang@...el.com>
To:     "Hansen, Dave" <dave.hansen@...el.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>
CC:     "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "Christopherson,, Sean" <seanjc@...gle.com>,
        "pbonzini@...hat.com" <pbonzini@...hat.com>,
        "Williams, Dan J" <dan.j.williams@...el.com>,
        "Wysocki, Rafael J" <rafael.j.wysocki@...el.com>,
        "kirill.shutemov@...ux.intel.com" <kirill.shutemov@...ux.intel.com>,
        "Huang, Ying" <ying.huang@...el.com>,
        "Chatre, Reinette" <reinette.chatre@...el.com>,
        "Brown, Len" <len.brown@...el.com>,
        "Luck, Tony" <tony.luck@...el.com>,
        "ak@...ux.intel.com" <ak@...ux.intel.com>,
        "Yamahata, Isaku" <isaku.yamahata@...el.com>,
        "Gao, Chao" <chao.gao@...el.com>,
        "sathyanarayanan.kuppuswamy@...ux.intel.com" 
        <sathyanarayanan.kuppuswamy@...ux.intel.com>,
        "david@...hat.com" <david@...hat.com>,
        "bagasdotme@...il.com" <bagasdotme@...il.com>,
        "Shahar, Sagi" <sagis@...gle.com>,
        "imammedo@...hat.com" <imammedo@...hat.com>
Subject: RE: [PATCH v9 07/18] x86/virt/tdx: Do TDX module per-cpu
 initialization

> On 2/13/23 03:59, Kai Huang wrote:
> > @@ -247,8 +395,17 @@ int tdx_enable(void)
> >  		ret = __tdx_enable();
> >  		break;
> >  	case TDX_MODULE_INITIALIZED:
> > -		/* Already initialized, great, tell the caller. */
> > -		ret = 0;
> > +		/*
> > +		 * The previous call of __tdx_enable() may only have
> > +		 * initialized part of present cpus during module
> > +		 * initialization, and new cpus may have become online
> > +		 * since then.
> > +		 *
> > +		 * To make sure all online cpus are TDX-runnable, always
> > +		 * do per-cpu initialization for all online cpus here
> > +		 * even the module has been initialized.
> > +		 */
> > +		ret = __tdx_enable_online_cpus();
> 
> I'm missing something here.  CPUs get initialized through either:
> 
>  1. __tdx_enable(), for the CPUs around at the time  2. tdx_cpu_online(), for
> hotplugged CPUs after __tdx_enable()
> 
> But, this is a third class.  CPUs that came online after #1, but which got missed
> by #2.  How can that happen?

(Replying via Microsoft Outlook cause my Evolution suddenly stopped to work after updating the Fedora).

Currently we depend on KVM's CPU hotplug callback to call tdx_cpu_online().  The problem is the KVM's callback can go away when KVM module gets unloaded.

For example:

	1) KVM module loaded when CPU 0, 1, 2 are online, CPU 3, 4, 5 are offline.
	2)  __tdx_enable() gets called.  LP.INIT are done on CPU 0, 1, 2.
	3) KVM gets unloaded.  It's CPU hotplug callbacks are removed too.
	4) CPU 3 becomes online.  In this case, tdx_cpu_online() is not called for it as the KVM's CPU hotplug callback is gone.

So later if KVM gets loaded again, we need to go through __tdx_enable_online_cpus() to do LP.INIT for CPU 3 as it's already online.

Perhaps I didn't explain clearly in the comment.  Below is the updated one:

		/*
		 * The previous call of __tdx_enable() may only have
		 * initialized part of present cpus during module
		 * initialization, and new cpus may have become online
		 * since then w/o doing per-cpu initialization.
		 * 
		 * For example, a new CPU can become online when KVM is
		 * unloaded, in which case tdx_cpu_enable() is not called since
		 * KVM's CPU online callback has been removed.
		 *
		 * To make sure all online cpus are TDX-runnable, always
		 * do per-cpu initialization for all online cpus here
		 * even the module has been initialized.
		 */