linux-kernel - RE: [x86] d55564cfc2: will-it-scale.per_thread

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <824461ae2cb642b1b2f82fac140a98da@AcuMS.aculab.com>
Date:   Fri, 8 Jan 2021 09:37:45 +0000
From:   David Laight <David.Laight@...LAB.COM>
To:     'Linus Torvalds' <torvalds@...ux-foundation.org>,
        Al Viro <viro@...iv.linux.org.uk>
CC:     kernel test robot <oliver.sang@...el.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...nel.org>, Borislav Petkov <bp@...en8.de>,
        Peter Zijlstra <peterz@...radead.org>,
        LKML <linux-kernel@...r.kernel.org>,
        "lkp@...ts.01.org" <lkp@...ts.01.org>,
        kernel test robot <lkp@...el.com>,
        "Huang, Ying" <ying.huang@...el.com>,
        Feng Tang <feng.tang@...el.com>,
        "zhengjun.xing@...el.com" <zhengjun.xing@...el.com>
Subject: RE: [x86] d55564cfc2: will-it-scale.per_thread_ops -5.8% regression

From: Linus Torvalds
> Sent: 07 January 2021 19:34
> 
> On Thu, Jan 7, 2021 at 11:04 AM Al Viro <viro@...iv.linux.org.uk> wrote:
> >
> > BTW, changing 'event' field in place from another thread is going to
> > be interesting - you have two 16bit values next to each other and
> > two CPUs modifying those with no exclusion.  Sounds like a recipe
> > for massive trouble...
> 
> It's perfectly fine on just about anything else than on an original
> pre-ev5 alpha.

Apart from the horrid cost of the cache-line bouncing.

> The C standard even - finally - made it a requirement that accesses to
> different members can't introduce data races.
> 
> So I agree with you that it's a bit annoying, and it's likely not even
> very common, but I could easily imagine myself writing code that
> changes either 'fd' or 'events' in a threaded server.
> 
> That's pretty much the whole point of 'poll()' after all - threaded
> servers that have that convenient array of pollable file descriptors.

I ended up using epoll().
One server thread does the epoll() and then all the threads process
the entries using atomic_increment() on the array index.

The lack of spinlocks in userspace really kills you.
If you use a futex to control a linked list a hardware interrupt
and then all the network and rcu softint callbacks can happen
in the few locked instrcutions.
It doesn't matter that one server thread is blocked for ~1ms,
but having them all blocked is a problem.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)