netdev - RE: [RFC PATCH 1/2] net: Add sysfs files for threaded NAPI.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <269e5fec0f0f4020a50ea23e535db63e@AcuMS.aculab.com>
Date: Thu, 25 May 2023 08:32:49 +0000
From: David Laight <David.Laight@...LAB.COM>
To: 'Sebastian Andrzej Siewior' <bigeasy@...utronix.de>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
CC: Eric Dumazet <edumazet@...gle.com>, Jakub Kicinski <kuba@...nel.org>,
	"Kurt Kanzenbach" <kurt.kanzenbach@...utronix.de>, Paolo Abeni
	<pabeni@...hat.com>, Thomas Gleixner <tglx@...utronix.de>
Subject: RE: [RFC PATCH 1/2] net: Add sysfs files for threaded NAPI.

From: Sebastian Andrzej Siewior
> Sent: 24 May 2023 12:13
> 
> I've been looking into threaded NAPI. One awkward thing to do is
> to figure out the thread names, pids in order to adjust the thread
> priorities and SMP affinity.
> On PREEMPT_RT the NAPI thread is treated (by the user) the same way as
> the threaded interrupt which means a dedicate CPU affinity for the
> thread and a higher task priority to be favoured over other tasks on the
> CPU. Otherwise the NAPI thread can be preempted by other threads leading
> to delays in packet delivery.

Dropped packets....

> Having to run ps/ grep is awkward to get the PID right. It is not easy
> to match the interrupt since there is no obvious relation between the
> IRQ and the NAPI thread.
> NAPI threads are enabled often to mitigate the problems caused by a
> "pending" ksoftirqd (which has been mitigated recently by doing softiqrs
> regardless of ksoftirqd status). There is still the part that the NAPI
> thread does not use softnet_data::poll_list.

I had to enable both threaded NAPI and RFS (splitting IP processing
to multiple threads) in order to avoid dropping ethernet packets.

To make it all work the NAPI threads processing the receive ring
had to be running under the RT scheduler.
None of the other NAPI threads need to be RT - and, indeed, it
is likely to be detrimental to run them under the RT scheduler.
(You pretty much need to limit the total number of RT threads
to the number of cpu cores so that the scheduler effectively
assigns a cpu to each RT thread.)
Trying to find the correct thread pids was definitely non-trivial.
Especially if you are trying to set it all up for an unknown system.
(As soon as you run ps | grep you are open to arbitrary process names.)

One of the problems with using softints for network receive is
that they suddenly drop from 'higher priority than any RT thread'
to 'a normal priority thread that might have its priority reduced'.
The latter is pretty useless for emptying network receive rings.

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)