netdev - Re: [PATCH net-next 1/2] net: napi: wake up ksoftirqd if needed after scheduling NAPI

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20220208161621.7b40619b@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Tue, 8 Feb 2022 16:16:21 -0800
From:   Jakub Kicinski <kuba@...nel.org>
To:     Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Cc:     Yannick Vignon <yannick.vignon@....nxp.com>,
        Eric Dumazet <edumazet@...gle.com>,
        Giuseppe Cavallaro <peppe.cavallaro@...com>,
        Alexandre Torgue <alexandre.torgue@...s.st.com>,
        Jose Abreu <joabreu@...opsys.com>,
        "David S. Miller" <davem@...emloft.net>,
        Maxime Coquelin <mcoquelin.stm32@...il.com>,
        Antoine Tenart <atenart@...nel.org>,
        Alexander Lobakin <alexandr.lobakin@...el.com>,
        Paolo Abeni <pabeni@...hat.com>, Wei Wang <weiwan@...gle.com>,
        Kumar Kartikeya Dwivedi <memxor@...il.com>,
        Yunsheng Lin <linyunsheng@...wei.com>,
        Arnd Bergmann <arnd@...db.de>, netdev <netdev@...r.kernel.org>,
        Vladimir Oltean <olteanv@...il.com>,
        Xiaoliang Yang <xiaoliang.yang_1@....com>, mingkai.hu@....com,
        Joakim Zhang <qiangqing.zhang@....com>,
        sebastien.laveze@....com, Yannick Vignon <yannick.vignon@....com>,
        Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH net-next 1/2] net: napi: wake up ksoftirqd if needed
 after scheduling NAPI

On Tue, 8 Feb 2022 18:45:53 +0100 Sebastian Andrzej Siewior wrote:
> On 2022-02-08 07:35:20 [-0800], Jakub Kicinski wrote:
> > > If you enable threaded NAPI then you end up with a thread and the
> > > softirq is no longer used. I don't know what the next action is but I
> > > guess you search for that thread and pin it manually to CPU and assign a
> > > RT priority (probably, otherwise it will compete with other tasks for
> > > CPU resources).  
> > 
> > FWIW I don't think servers would want RT prio.  
> 
> but then the NAPI thread is treated the same way like your xz -9.

You'd either pin the workload away from the network processing cores 
or, if there's no pinning, prefer requests to run to completion to
achieve lower latency.

> > > Instead we could have
> > > 	request_threaded_irq(, handler_irq, handler_napi);
> > > 
> > > And we would have basically the same outcome. Except that handler_napi()
> > > runs that SCHED_FIFO/50 and has the same CPU affinity as the IRQ (and
> > > the CPU affinity is adjusted if the IRQ-affinity is changed).
> > > We would still have to work out the details what handler_irq() is
> > > allowed to do and how to handle one IRQ and multiple handler_napi().
> > > 
> > > If you wrap request_threaded_irq() in something like request_napi_irq()
> > > the you could switch between the former (softirq) and later (thread)
> > > based NAPI handling (since you have all the needed details).  
> > 
> > One use case to watch out for is drivers which explicitly moved 
> > to threaded NAPI because they want to schedule multiple threads 
> > (NAPIs) from a single IRQ to spread processing across more cores.  
> 
> the request_napi_irq() could have a sysfs switch (like we currently
> have).
> But you mentioned one IRQ and multiple NAPI threads, to distribute
> across core. The usual case is that you have one IRQ for a network queue
> and this network queue has one NAPI struct, right?
> 
> In the case where you would have one IRQ but two network queues, each
> with one NAPI struct? And then you use the napi-threads and pin manually
> queue-napi#1 to CPU#1 and queue-napi#2 to CPU#2 while the IRQ itself
> fires on CPU#1?

It was mt76, the WiFi driver, I looked in the morning and I think it
had a NAPI for Tx and 2 NAPIs for Rx. The scheduler can spread them
around. Felix will have the exact details.