linux-kernel - Re: [PATCH v12 05/22] x86/virt/tdx: Add SEAMCALL infrastructure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <84f0acd416aef0796317e76edcf0b63622583436.camel@intel.com>
Date:   Wed, 28 Jun 2023 23:21:23 +0000
From:   "Huang, Kai" <kai.huang@...el.com>
To:     "peterz@...radead.org" <peterz@...radead.org>
CC:     "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "Raj, Ashok" <ashok.raj@...el.com>,
        "Hansen, Dave" <dave.hansen@...el.com>,
        "david@...hat.com" <david@...hat.com>,
        "bagasdotme@...il.com" <bagasdotme@...il.com>,
        "Luck, Tony" <tony.luck@...el.com>,
        "ak@...ux.intel.com" <ak@...ux.intel.com>,
        "Wysocki, Rafael J" <rafael.j.wysocki@...el.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Christopherson,, Sean" <seanjc@...gle.com>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "pbonzini@...hat.com" <pbonzini@...hat.com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "kirill.shutemov@...ux.intel.com" <kirill.shutemov@...ux.intel.com>,
        "Chatre, Reinette" <reinette.chatre@...el.com>,
        "Yamahata, Isaku" <isaku.yamahata@...el.com>,
        "nik.borisov@...e.com" <nik.borisov@...e.com>,
        "hpa@...or.com" <hpa@...or.com>, "Shahar, Sagi" <sagis@...gle.com>,
        "imammedo@...hat.com" <imammedo@...hat.com>,
        "bp@...en8.de" <bp@...en8.de>, "Gao, Chao" <chao.gao@...el.com>,
        "Brown, Len" <len.brown@...el.com>,
        "sathyanarayanan.kuppuswamy@...ux.intel.com" 
        <sathyanarayanan.kuppuswamy@...ux.intel.com>,
        "Huang, Ying" <ying.huang@...el.com>,
        "Williams, Dan J" <dan.j.williams@...el.com>,
        "x86@...nel.org" <x86@...nel.org>
Subject: Re: [PATCH v12 05/22] x86/virt/tdx: Add SEAMCALL infrastructure

On Wed, 2023-06-28 at 14:58 +0200, Peter Zijlstra wrote:
> On Tue, Jun 27, 2023 at 02:12:35AM +1200, Kai Huang wrote:
> 
> > +static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
> 
> __always_inline perhaps? __always_unused seems wrong, worse it's still
> there at the end of the series:
> 
> $ quilt diff --combine - | grep seamcall
> ...
> +static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
> ...
> +       ret = seamcall(TDH_SYS_INIT, 0, 0, 0, 0, NULL, NULL);
> +       ret = seamcall(TDH_SYS_LP_INIT, 0, 0, 0, 0, NULL, NULL);
> +       ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,
> +       ret = seamcall(TDH_SYS_CONFIG, __pa(tdmr_pa_array),
> +       return seamcall(TDH_SYS_KEY_CONFIG, 0, 0, 0, 0, NULL, NULL);
> +               ret = seamcall(TDH_SYS_TDMR_INIT, tdmr->base, 0, 0, 0, NULL,
> ...
> 
> Definitely not unused.

Thanks for reviewing!

Sorry obviously I forgot to remove __always_unused in the patch that firstly
used seamcall().  Should be more careful. :(

> 
> > +				    u64 *seamcall_ret,
> > +				    struct tdx_module_output *out)
> 
> This interface is atrocious :/ Why have these two ret values? Why can't
> that live in a single space -- /me looks throught the callers, and finds
> seamcall_ret is unused :-(

I'll @seamcall_ret as also suggested by Kirill.

> 
> Worse, the input (c,d,8,9) is a strict subset of the output
> (c,d,8,9,10,11) so why isn't that a single thing used for both input and
> output.
> 
> struct tdx_call {
> 	u64 rcx, rdx, r8, r9, r10, r11;
> };
> 
> static int __always_inline seamcall(u64 fn, struct tdx_call *regs)
> {
> }
> 
> 
> 	struct tdx_regs regs = { };
> 	ret = seamcall(THD_SYS_INIT, &regs);
> 
> 
> 
> 	struct tdx_regs regs = {
> 		.rcx = sysinfo_pa,	.rdx = TDXSYSINFO_STRUCT_SIZE,
> 		.r8  = cmr_array_pa,	.r9  = MAX_CMRS,
> 	};
> 	ret = seamcall(THD_SYS_INFO, &regs);
> 	if (ret)
> 		return ret;
> 
> 	print_cmrs(cmr_array, regs.r9);
> 
> 
> /me looks more at this stuff and ... WTF!?!?
> 
> Can someone explain to me why __tdx_hypercall() is sane (per the above)
> but then we grew __tdx_module_call() as an absolute abomination and are
> apparently using that for seam too?
> 
> 

Sorry I don't know the story behind __tdx_hypercall().

For TDCALL and SEAMCALL, I believe one reason is they can be used in performance
critical path.  The @out is not always used, so putting all outputs to a
structure can reduce the number of function parameters. I once had separate
struct tdx_seamcall_input {} and struct tdx_seamcall_out {} but wasn't
preferred.

Kirill, could you help to explain?

> 
> 
> > +{
> > +	u64 sret;
> > +	int cpu;
> > +
> > +	/* Need a stable CPU id for printing error message */
> > +	cpu = get_cpu();
> 
> And that's important because? 
> 

I want to have a stable cpu for error message printing.

> Does having preemption off across the
> seamcall make sense? Does it still make sense when you add a loop later?

SEAMCALL itself isn't interruptible, so I think having preemption off around
SEAMCALL is fine.  But I agree disabling preemption around multiple SEAMCALL
isn't ideal.  I'll change that to only disable preemption around one SEAMCALL to
get a correct CPU id for error printing.

> 
> > +	sret = __seamcall(fn, rcx, rdx, r8, r9, out);
> > +	put_cpu();
> > +
> > +	/* Save SEAMCALL return code if the caller wants it */
> > +	if (seamcall_ret)
> > +		*seamcall_ret = sret;
> > +
> > +	switch (sret) {
> > +	case 0:
> > +		/* SEAMCALL was successful */
> > +		return 0;
> > +	case TDX_SEAMCALL_VMFAILINVALID:
> > +		pr_err_once("module is not loaded.\n");
> > +		return -ENODEV;
> > +	default:
> > +		pr_err_once("SEAMCALL failed: CPU %d: leaf %llu, error 0x%llx.\n",
> > +				cpu, fn, sret);
> > +		if (out)
> > +			pr_err_once("additional output: rcx 0x%llx, rdx 0x%llx, r8 0x%llx, r9 0x%llx, r10 0x%llx, r11 0x%llx.\n",
> > +					out->rcx, out->rdx, out->r8,
> > +					out->r9, out->r10, out->r11);
> 
> At the very least this lacks { }, but it is quite horrendous coding
> style.
> 
> Why switch() at all, would not:
> 
> 	if (!rset)
> 		return 0;
> 
> 	if (sret == TDX_SEAMCALL_VMFAILINVALID) {
> 		pr_nonsense();
> 		return -ENODEV;
> 	}
> 
> 	if (sret == TDX_SEAMCALL_GP) {
> 		pr_nonsense();
> 		return -ENODEV;
> 	}
> 
> 	if (sret == TDX_SEAMCALL_UD) {
> 		pr_nonsense();
> 		return -EINVAL;
> 	}
> 
> 	pr_nonsense();
> 	return -EIO;
> 
> be much clearer and have less horrific indenting issues?

I can certainly change to this style.  Thanks.