netdev - Re: [PATCH net-next] net: rps: softnet_data reorg to make enqueue_to

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAAVpQUDRgnuxf5gcX21AfBzr6MbcTL7UJgkfGgAMb2zHdYXOxw@mail.gmail.com>
Date: Mon, 27 Oct 2025 10:18:19 -0700
From: Kuniyuki Iwashima <kuniyu@...gle.com>
To: Eric Dumazet <edumazet@...gle.com>
Cc: "David S . Miller" <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>, 
	Paolo Abeni <pabeni@...hat.com>, Simon Horman <horms@...nel.org>, netdev@...r.kernel.org, 
	eric.dumazet@...il.com
Subject: Re: [PATCH net-next] net: rps: softnet_data reorg to make
 enqueue_to_backlog() fast

On Fri, Oct 24, 2025 at 2:12 AM Eric Dumazet <edumazet@...gle.com> wrote:
>
> enqueue_to_backlog() is showing up in kernel profiles on hosts
> with many cores, when RFS/RPS is used.
>
> The following softnet_data fields need to be updated:
>
> - input_queue_tail
> - input_pkt_queue (next, prev, qlen, lock)
> - backlog.state (if input_pkt_queue was empty)
>
> Unfortunately they are currenly using two cache lines:
>
>         /* --- cacheline 3 boundary (192 bytes) --- */
>         call_single_data_t         csd __attribute__((__aligned__(64))); /*  0xc0  0x20 */
>         struct softnet_data *      rps_ipi_next;         /*  0xe0   0x8 */
>         unsigned int               cpu;                  /*  0xe8   0x4 */
>         unsigned int               input_queue_tail;     /*  0xec   0x4 */
>         struct sk_buff_head        input_pkt_queue;      /*  0xf0  0x18 */
>
>         /* --- cacheline 4 boundary (256 bytes) was 8 bytes ago --- */
>
>         struct napi_struct         backlog __attribute__((__aligned__(8))); /* 0x108 0x1f0 */
>
> Add one ____cacheline_aligned_in_smp to make sure they now are using
> a single cache line.
>
> Also, because napi_struct has written fields, make @state its first field.
>
> We want to make sure that cpus adding packets to sd->input_pkt_queue
> are not slowing down cpus processing their backlog because of
> false sharing.
>
> After this patch new layout is:
>
>         /* --- cacheline 5 boundary (320 bytes) --- */
>         long int                   pad[3] __attribute__((__aligned__(64))); /* 0x140  0x18 */
>         unsigned int               input_queue_tail;     /* 0x158   0x4 */
>
>         /* XXX 4 bytes hole, try to pack */
>
>         struct sk_buff_head        input_pkt_queue;      /* 0x160  0x18 */
>         struct napi_struct         backlog __attribute__((__aligned__(8))); /* 0x178 0x1f0 */
>
> Signed-off-by: Eric Dumazet <edumazet@...gle.com>

Reviewed-by: Kuniyuki Iwashima <kuniyu@...gle.com>