netdev - Re: [PATCH v1] x86/csum: rewrite csum

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sun, 14 Nov 2021 07:03:58 -0800
From:   Eric Dumazet <edumazet@...gle.com>
To:     David Laight <David.Laight@...lab.com>
Cc:     Alexander Duyck <alexander.duyck@...il.com>,
        Eric Dumazet <eric.dumazet@...il.com>,
        "David S . Miller" <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>,
        netdev <netdev@...r.kernel.org>,
        "the arch/x86 maintainers" <x86@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>
Subject: Re: [PATCH v1] x86/csum: rewrite csum_partial()

On Sun, Nov 14, 2021 at 6:44 AM David Laight <David.Laight@...lab.com> wrote:
>
> From: Eric Dumazet
> > Sent: 11 November 2021 22:31
> ..
> > That requires an extra add32_with_carry(), which unfortunately made
> > the thing slower for me.
> >
> > I even hardcoded an inline fast_csum_40bytes() and got best results
> > with the 10+1 addl,
> > instead of
> >  (5 + 1) acql +  mov (needing one extra  register) + shift + addl + adcl
>
> Did you try something like:
>         sum = buf[0];
>         val = buf[1]:
>         asm(
>                 add64 sum, val
>                 adc64 sum, buf[2]
>                 adc64 sum, buf[3]
>                 adc64 sum, buf[4]
>                 adc64 sum, 0
>         }
>         sum_hi = sum >> 32;
>         asm(
>                 add32 sum, sum_hi
>                 adc32 sum, 0
>         )

This is what I tried. but the last part was using add32_with_carry(),
and clang was adding stupid mov to temp variable on the stack,
killing the perf.

This issue was solved with

diff --git a/arch/x86/include/asm/checksum_64.h
b/arch/x86/include/asm/checksum_64.h
index 9af3aed54c6b945e1216719db6889d38ef47cce7..56981943d49cbaa934f7dbac9afb1f575a2437b9
100644
--- a/arch/x86/include/asm/checksum_64.h
+++ b/arch/x86/include/asm/checksum_64.h
@@ -174,7 +174,7 @@ static inline unsigned add32_with_carry(unsigned
a, unsigned b)
        asm("addl %2,%0\n\t"
            "adcl $0,%0"
            : "=r" (a)
-           : "0" (a), "rm" (b));
+           : "0" (a), "r" (b));
        return a;
 }



> Splitting it like that should allow thew compiler to insert
> additional instructions between the two 'adc' blocks
> making it much more likely that the cpu will schedule them
> in parallel with other instructions.
>
> The extra 5 adc32 have to add 5 clocks (register dependency chain).
> The 'mov' ought to be free (register rename) and the extra shift
> and adds one clock each - so 3 (maybe 4) clocks.
> So the 64bit version really ought to be faster even a s single
> asm block.
>
> Trying to second-guess the x86 cpu is largely impossible :-)
>
> Oh, and then try the benchmarks of one of the 64bit Atom cpus
> used in embedded systems....
> We've some 4core+hyperthreading ones that aren't exactly slow.
>
>         David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)