linux-kernel - Re: [RFC 1/2] softirq: Defer net rx/tx processing to ksoftirqd context

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANn89i+ehJg_7YfOCicgv_EuQWR6Xn7GHi+g5=atigeXDeNMHw@mail.gmail.com>
Date:   Thu, 11 Jan 2018 12:16:42 -0800
From:   Eric Dumazet <edumazet@...gle.com>
To:     Linus Torvalds <torvalds@...ux-foundation.org>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Dmitry Safonov <dima@...sta.com>,
        Frederic Weisbecker <frederic@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Dmitry Safonov <0x7f454c46@...il.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        David Miller <davem@...emloft.net>,
        Frederic Weisbecker <fweisbec@...il.com>,
        Hannes Frederic Sowa <hannes@...essinduktion.org>,
        Ingo Molnar <mingo@...nel.org>,
        "Levin, Alexander (Sasha Levin)" <alexander.levin@...izon.com>,
        Paolo Abeni <pabeni@...hat.com>,
        "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
        Radu Rendec <rrendec@...sta.com>,
        Rik van Riel <riel@...hat.com>,
        Stanislaw Gruszka <sgruszka@...hat.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Wanpeng Li <wanpeng.li@...mail.com>
Subject: Re: [RFC 1/2] softirq: Defer net rx/tx processing to ksoftirqd context

On Thu, Jan 11, 2018 at 12:03 PM, Linus Torvalds
<torvalds@...ux-foundation.org> wrote:
> On Thu, Jan 11, 2018 at 11:48 AM, Eric Dumazet <edumazet@...gle.com> wrote:
>> That was the purpose on the last patch : As soon as ksoftirqd is scheduled
>> (by some kind of jitter in the 99,000 pps workload, or antagonist wakeup),
>> we then switch to a mode where process scheduler can make decisions
>> based on threads prios and cpu usage.
>
> Yeah, but that really screws up everybody else.
>
> It really is a soft *interrupt*. That was what it was designed for.
> The thread handling is not primary, it's literally a fallback to avoid
> complete starvation.
>
> The fact that networking has now - for several years - tried to make
> it some kind of thread and get fairness with user threads is all
> entirely antithetical to what softirq was designed for.
>
>> Then, as soon as the load was able to finish in its quantum the
>> pending irqs, we re-enter the mode
>> where softirq are immediately serviced.
>
> Except that's not at all how the code works.
>
> As I pointed out, the softirq thread can be scheduled away, but the
> "softiq_running()" wil stilll return true - and the networking code
> has now screwed up all the *other* softirqs too!
>
> I really suspect that what networking wants is more like the
> workqueues. Or at least more isolation between different softirq
> users, but that's fairly fundamentally hard, given how softirqs are
> designed.
>

Note that when I implemented TCP Small queues, I did experiments between
using a work queue or a tasklet, and workqueues added unacceptable P99
latencies,
when many user threads are competing with kernel threads.

I suspect that firing a worqueue for networking RX will likely have
the same effect :/

Note that the current __do_softirq() implementation suffers from the following :

Say we receive NET_RX softirq
-> While processing the packet, we wakeup on thread (thus
need_resched() becomes true),
but also raise a tasklet (because a particular driver needs some extra
processing in tasklet context instead of NET_RX ???)

-> Then we exit __do_softirq() _and_ schedule ksoftirqd (because
tasklet  needs to be serviced)