lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <434810d9-1b36-496d-a10d-41c6c068375e@intel.com>
Date: Mon, 2 Jun 2025 12:12:51 -0700
From: "Chang S. Bae" <chang.seok.bae@...el.com>
To: Chao Gao <chao.gao@...el.com>, Dave Hansen <dave.hansen@...el.com>
CC: Sean Christopherson <seanjc@...gle.com>, <x86@...nel.org>,
	<linux-kernel@...r.kernel.org>, <kvm@...r.kernel.org>, <tglx@...utronix.de>,
	<pbonzini@...hat.com>, <peterz@...radead.org>, <rick.p.edgecombe@...el.com>,
	<weijiang.yang@...el.com>, <john.allen@....com>, <bp@...en8.de>,
	<xin3.li@...el.com>, Dave Hansen <dave.hansen@...ux.intel.com>, Eric Biggers
	<ebiggers@...gle.com>, "H. Peter Anvin" <hpa@...or.com>, Ingo Molnar
	<mingo@...hat.com>, Kees Cook <kees@...nel.org>, Maxim Levitsky
	<mlevitsk@...hat.com>, Mitchell Levy <levymitchell0@...il.com>, "Nikolay
 Borisov" <nik.borisov@...e.com>, Oleg Nesterov <oleg@...hat.com>, Sohil Mehta
	<sohil.mehta@...el.com>, Stanislav Spassov <stanspas@...zon.de>, "Vignesh
 Balasubramanian" <vigbalas@....com>
Subject: Re: [PATCH v8 0/6] Introduce CET supervisor state support

On 5/27/2025 4:01 AM, Chao Gao wrote:
> 
> *: https://lore.kernel.org/all/88cb75d3-01b9-38ea-e29f-b8fefb548573@intel.com/
> 
> The issue arises because the XFD MSR retains the value (i.e., 0, indicating
> AMX enabled) from the previous process, while both the passed-in fpstate
> (init_fpstate) and the current fpstate have AMX disabled.
> 
> To reproduce this issue, compile the kernel with CONFIG_PREEMPT=y, apply the
> attached diff to the amx selftest and run:
> 
> # numactl -C 1 ./tools/testing/selftests/x86/amx_64
> 
> diff --git a/tools/testing/selftests/x86/amx.c b/tools/testing/selftests/x86/amx.c
> index 40769c16de1b..4d533d1a530d 100644
> --- a/tools/testing/selftests/x86/amx.c
> +++ b/tools/testing/selftests/x86/amx.c
> @@ -430,6 +430,10 @@ static inline void validate_tiledata_regs_changed(struct xsave_buffer *xbuf)
> 		fatal_error("TILEDATA registers did not change");
>   }
>   
> +static void dummy_handler(int sig)
> +{
> +}
> +
>   /* tiledata inheritance test */
>   
>   static void test_fork(void)
> @@ -444,6 +448,10 @@ static void test_fork(void)
> 		/* fork() succeeded.  Now in the parent. */
> 		int status;
>   
> +		req_xtiledata_perm();
> +		load_rand_tiledata(stashed_xsave);
> +		while(1);
> +
> 		wait(&status);
> 		if (!WIFEXITED(status) || WEXITSTATUS(status))
> 			fatal_error("fork test child");
> @@ -452,7 +460,9 @@ static void test_fork(void)
> 	/* fork() succeeded.  Now in the child. */
> 	printf("[RUN]\tCheck tile data inheritance.\n\tBefore fork(), load tiledata\n");
>   
> -	load_rand_tiledata(stashed_xsave);
> +	signal(SIGSEGV, dummy_handler);
> +	while(1)
> +		raise(SIGSEGV);
>   
> 	grandchild = fork();
> 	if (grandchild < 0) {
> @@ -500,9 +510,6 @@ int main(void)
>   
> 	test_dynamic_state();
>   
> -	/* Request permission for the following tests */
> -	req_xtiledata_perm();
> -
> 	test_fork();
>   
> 	/*

The test case creates two processes -- the first uses AMX (task #1), and 
the other continuously sends signals without using AMX (task #2).

This leads to task #2 being preempted by task #1.

This behavior aligns with Sean’s report. From my investigation, the 
issue appears to have existed for quite some time and is not related to 
the changes in this series.

Here’s a summary of my findings:

== Preempt Case ==

To illustrate how the XFD MSR state becomes incorrect in this scenario:

  task #1 (fpstate->xfd=0)  task #2 (fpstate->xfd=0x80000)
  ========================  ==============================
                            handle_signal()
                            -> setup_rt_frame()
                               -> get_siframe()
                                  -> copy_fpstate_to_sigframe()
                                     -> fpregs_unlock()
                                  ...
   ...
   switch_fpu_return()
   -> fpregs_restore_userregs()
      -> restore_fpregs_from_fpstate()
         -> xfd_write_state()
            ^ IA32_XFD_MSR = 0
   ...
                                  ...
                               -> fpu__clear_user_states()
                                  -> fpregs_lock()
                                  -> restore_fpregs_from_init_fpstate()
                                     -> os_rstor()
                                        -> xfd_validate_state()
                                           ^ IA32_XFD_MSR != fpstate->xfd
                                  -> fpregs_mark_active()
                                  -> fpregs_unlock()

Since fpu__clear_user_states() marks the FPU state as valid in the end, 
an XFD MSR sync-up was clearly missing.

== Return-to-Userspace Path ==

Both tasks at that moment are on the return-to-userspace path, but at 
different points in IRQ state:

   * task #2 is inside handle_signal() and already re-enabled IRQs.
   * task #1 is after IRQ is disabled again when calling
     switch_fpu_return().

   local_irq_disable_exit_to_user()
   exit_to_user_mode_prepare()
   -> exit_to_user_mode_loop()
      -> local_irq_enable_exit_to_user()
         -> arch_do_signal_or_restart()
            -> handle_signal()
      -> local_irq_disable_exit_to_user()
   -> arch_exit_user_mode_prepare()
      -> arch_exit_work()
         -> switch_fpu_return()

This implies that fpregs_lock()/fpregs_unlock() is necessary inside 
handle_signal() when XSAVE instructions are invoked.

But, it should be okay for switch_fpu_return() to call 
fpregs_restore_userregs() without fpregs_lock().

== XFD Sanity Checker ==

The XFD sanity checker -- xfd_op_valid() -- correctly caught this issue 
in the test case. However, it may have a false negative when AMX usage 
was flipped between the two tasks.

Despite that, I don't think extending its coverage is worthwhile, as it 
would complicate the logic. The current logic and documentation seem 
sound.

== Fix Consideration ==

I think the fix is straightforward: resynchronize the IA32_XFD MSR in 
fpu__clear_user_states().

The existing xfd_update_state() function is self-contained and already 
performs feature checks and conditional MSR updates. Thus, it is not 
necessary to check the TIF_NEED_FPU_LOAD flag for this.

On the other hand, the sigreturn path already performs the XFD resync 
introduced by this commit:

   672365477ae8a ("x86/fpu: Update XFD state where required")

But I think that change was supposed to cover _both_ signal return and 
signal delivery paths. Sorry, it seems I had overlooked the latter
before.

Thanks,
Chang

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ