| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <ZG4TRruF/p9C/ZNu@corigine.com>
Date: Wed, 24 May 2023 15:38:14 +0200
From: Simon Horman <simon.horman@...igine.com>
To: Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Cc: netdev@...r.kernel.org, Eric Dumazet <edumazet@...gle.com>,
Jakub Kicinski <kuba@...nel.org>,
Kurt Kanzenbach <kurt.kanzenbach@...utronix.de>,
Paolo Abeni <pabeni@...hat.com>,
Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [RFC PATCH 1/2] net: Add sysfs files for threaded NAPI.
On Wed, May 24, 2023 at 01:12:58PM +0200, Sebastian Andrzej Siewior wrote:
> I've been looking into threaded NAPI. One awkward thing to do is
> to figure out the thread names, pids in order to adjust the thread
> priorities and SMP affinity.
> On PREEMPT_RT the NAPI thread is treated (by the user) the same way as
> the threaded interrupt which means a dedicate CPU affinity for the
> thread and a higher task priority to be favoured over other tasks on the
> CPU. Otherwise the NAPI thread can be preempted by other threads leading
> to delays in packet delivery.
> Having to run ps/ grep is awkward to get the PID right. It is not easy
> to match the interrupt since there is no obvious relation between the
> IRQ and the NAPI thread.
> NAPI threads are enabled often to mitigate the problems caused by a
> "pending" ksoftirqd (which has been mitigated recently by doing softiqrs
> regardless of ksoftirqd status). There is still the part that the NAPI
> thread does not use softnet_data::poll_list.
>
> To make things easier to setup NAPI threads here is a sysfs interfaces.
> It provides for each NAPI instance a folder containing the name and PID
> of the NAPI thread and an interrupt number of the interrupt scheduling
> the NAPI thread. The latter requires support from the driver.
> The name of the napi-instance can also be set by driver so it does not
> fallback to the NAPI-id.
>
> I've been thinking to wire up task affinity to follow the affinity of
> the interrupt thread. While this would require some extra work, it
> shouldn't be needed since the PID of the NAPI thread and interrupt
> number is exposed so the user may use chrt/ taskset to adjust the
> priority accordingly and the interrupt affinity does not change
> magically.
>
> Having said all that, there is still no generic solution to the
> "overload" problem. Part of the problem is lack of policy since the
> offload to ksoftirqd is not welcomed by everyone. Also "better" cards
> support filtering by ether type which allows to filter the problematic
> part to another NAPI instance avoiding the prbolem.
>
> This is what the structure looks with the igb driver after adding the
> name/ irq hints (second patch).
>
> | root@box:/sys/class/net# cd eno0
> | root@box:/sys/class/net/eno0# ls -l napi
> | total 0
>
> Empty before threaded NAPI is enabled.
>
> | root@box:/sys/class/net/eno0# echo 1 > threaded
> | root@box:/sys/class/net/eno0# ls -l napi
> | total 0
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-0
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-1
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-2
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-3
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-4
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-5
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-6
> | drwxr-xr-x 2 root root 0 May 24 09:42 eno0-TxRx-7
>
> Deployed using names supplied by the driver which map the names which
> are used for the IRQ.
>
> | root@box:/sys/class/net/eno0# grep . napi/*/*
> | napi/eno0-TxRx-0/interrupt:37
> | napi/eno0-TxRx-0/name:napi/eno0-8193
> | napi/eno0-TxRx-0/pid:2253
> | napi/eno0-TxRx-1/interrupt:38
> | napi/eno0-TxRx-1/name:napi/eno0-8194
> | napi/eno0-TxRx-1/pid:2252
> | napi/eno0-TxRx-2/interrupt:39
> | napi/eno0-TxRx-2/name:napi/eno0-8195
> | napi/eno0-TxRx-2/pid:2251
> | napi/eno0-TxRx-3/interrupt:40
> | napi/eno0-TxRx-3/name:napi/eno0-8196
> | napi/eno0-TxRx-3/pid:2250
> | napi/eno0-TxRx-4/interrupt:41
> | napi/eno0-TxRx-4/name:napi/eno0-8197
> | napi/eno0-TxRx-4/pid:2249
> | napi/eno0-TxRx-5/interrupt:42
> | napi/eno0-TxRx-5/name:napi/eno0-8198
> | napi/eno0-TxRx-5/pid:2248
> | napi/eno0-TxRx-6/interrupt:43
> | napi/eno0-TxRx-6/name:napi/eno0-8199
> | napi/eno0-TxRx-6/pid:2247
> | napi/eno0-TxRx-7/interrupt:44
> | napi/eno0-TxRx-7/name:napi/eno0-8200
> | napi/eno0-TxRx-7/pid:2246
>
> Thread name, pid and interrupt number as provided by the driver.
>
> | root@box:/sys/class/net/eno0# grep eno0-TxRx-7 /proc/interrupts | sed 's@ \+@ @g'
> | 44: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 123 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:07:00.0 8-edge eno0-TxRx-7
> | root@box:/sys/class/net/eno0# cat /proc/irq/44/smp_affinity_list
> | 0-7,16-23
> | root@box:/sys/class/net/eno0# cat /proc/irq/44/effective_affinity_list
> | 18
>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Hi Sebastian,
thanks for your patch.
A few nits from my side.
...
> @@ -2411,6 +2414,7 @@ struct net_device {
> struct rtnl_hw_stats64 *offload_xstats_l3;
>
> struct devlink_port *devlink_port;
> + struct kset *napi_kset;
> };
nit: Please add napi_kset to the kdoc for struct net_device
which is immediately above it.
...
> +int napi_thread_add_kobj(struct napi_struct *n)
> +{
> + struct kobject *kobj;
> + char napi_name[32];
> + const char *name_ptr;
nit: Please use reverse xmas tree order - longest line to shortest - for
networking code.
> + int ret;
> +
> + if (n->napi_name) {
> + name_ptr = n->napi_name;
> + } else {
> + name_ptr = napi_name;
> + scnprintf(napi_name, sizeof(napi_name), "napi_id-%d", n->napi_id);
> + }
> + kobj = &n->kobj;
> + kobj->kset = n->dev->napi_kset;
> + ret = kobject_init_and_add(kobj, &napi_ktype, NULL,
> + name_ptr);
> + return ret;
> +}
...
Powered by blists - more mailing lists