linux-kernel - Re: [PATCH] x86/crc32: optimize tail handling for crc32c short inputs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAFULd4Y8=zov8Yov3n4rhwT7JO0YxaEQJ56hq1Hd7TB=yQjWQw@mail.gmail.com>
Date: Wed, 5 Mar 2025 11:10:40 +0100
From: Uros Bizjak <ubizjak@...il.com>
To: Eric Biggers <ebiggers@...nel.org>
Cc: linux-kernel@...r.kernel.org, Bill Wendling <morbo@...gle.com>, 
	Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>, 
	Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org, 
	"H . Peter Anvin" <hpa@...or.com>, Ard Biesheuvel <ardb@...nel.org>, Nathan Chancellor <nathan@...nel.org>, 
	Nick Desaulniers <nick.desaulniers+lkml@...il.com>, Justin Stitt <justinstitt@...gle.com>, 
	David Laight <david.laight.linux@...il.com>, linux-crypto@...r.kernel.org, 
	llvm@...ts.linux.dev
Subject: Re: [PATCH] x86/crc32: optimize tail handling for crc32c short inputs

On Wed, Mar 5, 2025 at 10:26 AM Eric Biggers <ebiggers@...nel.org> wrote:
>
> From: Eric Biggers <ebiggers@...gle.com>
>
> For handling the 0 <= len < sizeof(unsigned long) bytes left at the end,
> do a 4-2-1 step-down instead of a byte-at-a-time loop.  This allows
> taking advantage of wider CRC instructions.  Note that crc32c-3way.S
> already uses this same optimization too.
>
> crc_kunit shows an improvement of about 25% for len=127.
>
> Suggested-by: H. Peter Anvin <hpa@...or.com>
> Signed-off-by: Eric Biggers <ebiggers@...gle.com>

LGTM.

Acked-by: Uros Bizjak <ubizjak@...il.com>

Thanks,
Uros.

> ---
>
> This applies to
> https://web.git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=crc-next
>
>  arch/x86/lib/crc32-glue.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/lib/crc32-glue.c b/arch/x86/lib/crc32-glue.c
> index 4b4721176799a..e3f93b17ac3f1 100644
> --- a/arch/x86/lib/crc32-glue.c
> +++ b/arch/x86/lib/crc32-glue.c
> @@ -55,11 +55,19 @@ u32 crc32c_arch(u32 crc, const u8 *p, size_t len)
>
>         for (num_longs = len / sizeof(unsigned long);
>              num_longs != 0; num_longs--, p += sizeof(unsigned long))
>                 asm(CRC32_INST : "+r" (crc) : ASM_INPUT_RM (*(unsigned long *)p));
>
> -       for (len %= sizeof(unsigned long); len; len--, p++)
> +       if (sizeof(unsigned long) > 4 && (len & 4)) {
> +               asm("crc32l %1, %0" : "+r" (crc) : ASM_INPUT_RM (*(u32 *)p));
> +               p += 4;
> +       }
> +       if (len & 2) {
> +               asm("crc32w %1, %0" : "+r" (crc) : ASM_INPUT_RM (*(u16 *)p));
> +               p += 2;
> +       }
> +       if (len & 1)
>                 asm("crc32b %1, %0" : "+r" (crc) : ASM_INPUT_RM (*p));
>
>         return crc;
>  }
>  EXPORT_SYMBOL(crc32c_arch);
>
> base-commit: 13f3d13d88b5dcba104a204fcbee61c75f8407d0
> --
> 2.48.1
>