linux-kernel - Re: [PATCH 2/2] x86/bitops: Fix false output register dependency of TZCNT insn

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <75C56CF0-A414-4AB8-BCD9-29CE5A344166@zytor.com>
Date: Tue, 25 Mar 2025 11:29:47 -0700
From: "H. Peter Anvin" <hpa@...or.com>
To: Uros Bizjak <ubizjak@...il.com>, x86@...nel.org,
        linux-kernel@...r.kernel.org
CC: Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...nel.org>,
        Borislav Petkov <bp@...en8.de>,
        Dave Hansen <dave.hansen@...ux.intel.com>
Subject: Re: [PATCH 2/2] x86/bitops: Fix false output register dependency of TZCNT insn

On March 25, 2025 10:52:02 AM PDT, Uros Bizjak <ubizjak@...il.com> wrote:
>On Haswell and later Intel processors, the TZCNT instruction appears
>to have a false dependency on the destination register. Even though
>the instruction only writes to it, the instruction will wait until
>destination is ready before executing. This false dependency
>was fixed for Skylake (and later) processors.
>
>Fix false dependency by clearing the destination register first.
>
>The x86_64 defconfig object size increases by 4215 bytes:
>
>	    text           data     bss      dec            hex filename
>	27342396        4642999  814852 32800247        1f47df7 vmlinux-old.o
>	27346611        4643015  814852 32804478        1f48e7e vmlinux-new.o
>
>Signed-off-by: Uros Bizjak <ubizjak@...il.com>
>Cc: Thomas Gleixner <tglx@...utronix.de>
>Cc: Ingo Molnar <mingo@...nel.org>
>Cc: Borislav Petkov <bp@...en8.de>
>Cc: Dave Hansen <dave.hansen@...ux.intel.com>
>Cc: "H. Peter Anvin" <hpa@...or.com>
>---
> arch/x86/include/asm/bitops.h | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
>diff --git a/arch/x86/include/asm/bitops.h b/arch/x86/include/asm/bitops.h
>index bbaf75ea6703..7e3d1cc97c5a 100644
>--- a/arch/x86/include/asm/bitops.h
>+++ b/arch/x86/include/asm/bitops.h
>@@ -248,8 +248,9 @@ arch_test_bit_acquire(unsigned long nr, const volatile unsigned long *addr)
> 
> static __always_inline unsigned long variable__ffs(unsigned long word)
> {
>-	asm("tzcnt %1,%0"
>-		: "=r" (word)
>+	asm("xor %k0,%k0\n\t" /* avoid false dependency on dest register */
>+	    "tzcnt %1,%0"
>+		: "=&r" (word)
> 		: ASM_INPUT_RM (word));
> 	return word;
> }
>@@ -267,8 +268,9 @@ static __always_inline unsigned long variable__ffs(unsigned long word)
> 
> static __always_inline unsigned long variable_ffz(unsigned long word)
> {
>-	asm("tzcnt %1,%0"
>-		: "=r" (word)
>+	asm("xor %k0,%k0\n\t" /* avoid false dependency on dest register */
>+	    "tzcnt %1,%0"
>+		: "=&r" (word)
> 		: "r" (~word));
> 	return word;
> }

Is xor better there than a mov (making it a copy of the source)? It might be more frequently fusable.