netdev - Re: [PATCH v2] net: add support for threaded NAPI polling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200806125708.6492ebfe@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Thu, 6 Aug 2020 12:57:08 -0700
From:   Jakub Kicinski <kuba@...nel.org>
To:     Eric Dumazet <eric.dumazet@...il.com>
Cc:     Felix Fietkau <nbd@....name>, netdev@...r.kernel.org,
        Hillf Danton <hdanton@...a.com>
Subject: Re: [PATCH v2] net: add support for threaded NAPI polling

On Thu, 6 Aug 2020 12:25:08 -0700 Eric Dumazet wrote:
> On 8/6/20 11:55 AM, Jakub Kicinski wrote:
> > I'm still trying to wrap my head around this.
> > 
> > Am I understanding correctly that you have one IRQ and multiple NAPI
> > instances?
> > 
> > Are we not going to end up with pretty terrible cache locality here if
> > the scheduler starts to throw rx and tx completions around to random
> > CPUs?
> > 
> > I understand that implementing separate kthreads would be more LoC, but
> > we do have ksoftirqs already... maybe we should make the NAPI ->
> > ksoftirq mapping more flexible, and improve the logic which decides to
> > load ksoftirq rather than make $current() pay?
> > 
> > Sorry for being slow.
> 
> Issue with ksoftirqd is that
> - it is bound to a cpu

Do you envision the scheduler balancing or work stealing being
advantageous in some configurations?

I was guessing that for compute workloads having ksoftirq bound will
actually make things more predictable/stable.

For pure routers (where we expect multiple cores to reach 100% just
doing packet forwarding) as long as there is an API to re-balance NAPIs
to cores - a simple specialized user space daemon would probably do a
better job as it can consult packet drop metrics etc.

Obviously I have no data to back up these claims..

> - Its nice value is 0, meaning that user threads can sometime compete too much with it.

True, I thought we could assume user level tuning.

> - It handles all kinds of softirqs, so messing with it might hurt some other layer.

Right, I have no data on how much this hurts in practice.

> Note that the patch is using a dedicate work queue. It is going to be not practical
> in case you need to handle two different NIC, and want separate pools for each of them.
> 
> Ideally, having one kthread per queue would be nice, but then there is more plumbing
> work to let these kthreads being visible in a convenient way (/sys/class/net/ethX/queues/..../kthread)

Is context switching cost negligible?

ksoftirq-like thread replicates all the NAPI budget-level mixing we
already do today.