netdev - Re: [RFC net-next 0/6] Cleanup IRQ affinity checks in several drivers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ZsRUvcOKLnTw16wD@LQ3V64L9R2.home>
Date: Tue, 20 Aug 2024 09:33:01 +0100
From: Joe Damato <jdamato@...tly.com>
To: Shay Drori <shayd@...dia.com>
Cc: Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org,
	Daniel Borkmann <daniel@...earbox.net>,
	"David S. Miller" <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>,
	Harshitha Ramamurthy <hramamurthy@...gle.com>,
	"moderated list:INTEL ETHERNET DRIVERS" <intel-wired-lan@...ts.osuosl.org>,
	Jeroen de Borst <jeroendb@...gle.com>,
	Jiri Pirko <jiri@...nulli.us>, Leon Romanovsky <leon@...nel.org>,
	open list <linux-kernel@...r.kernel.org>,
	"open list:MELLANOX MLX4 core VPI driver" <linux-rdma@...r.kernel.org>,
	Lorenzo Bianconi <lorenzo@...nel.org>,
	Paolo Abeni <pabeni@...hat.com>,
	Praveen Kaligineedi <pkaligineedi@...gle.com>,
	Przemek Kitszel <przemyslaw.kitszel@...el.com>,
	Saeed Mahameed <saeedm@...dia.com>,
	Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
	Shailend Chand <shailend@...gle.com>,
	Tariq Toukan <tariqt@...dia.com>,
	Tony Nguyen <anthony.l.nguyen@...el.com>,
	Willem de Bruijn <willemb@...gle.com>,
	Yishai Hadas <yishaih@...dia.com>,
	Ziwei Xiao <ziweixiao@...gle.com>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [RFC net-next 0/6] Cleanup IRQ affinity checks in several drivers

On Tue, Aug 20, 2024 at 09:40:31AM +0300, Shay Drori wrote:
> 
> 
> On 15/08/2024 13:22, Joe Damato wrote:
> > External email: Use caution opening links or attachments
> > 
> > 
> > On Wed, Aug 14, 2024 at 05:20:46PM -0700, Jakub Kicinski wrote:
> > > On Wed, 14 Aug 2024 19:01:40 +0100 Joe Damato wrote:
> > > > If it is, then the only option is to have the drivers pass in their
> > > > IRQ affinity masks, as Stanislav suggested, to avoid adding that
> > > > call to the hot path.
> > > > 
> > > > If not, then the IRQ from napi_struct can be used and the affinity
> > > > mask can be generated on every napi poll. i40e/gve/iavf would need
> > > > calls to netif_napi_set_irq to set the IRQ mapping, which seems to
> > > > be straightforward.
> > > 
> > > It's a bit sad to have the generic solution blocked.
> > > cpu_rmap_update() is exported. Maybe we can call it from our notifier?
> > > rmap lives in struct net_device
> > 
> > I agree on the sadness. I will take a look today.
> > 
> > I guess if we were being really ambitious, we'd try to move ARFS
> > stuff into the core (as RSS was moved into the core).
> 
> 
> Sorry for the late reply. Maybe we can modify affinity notifier infra to
> support more than a single notifier per IRQ.
> @Thomas, do you know why only a single notifier per IRQ is supported?

Sorry for the delayed response as well on my side; I've been in
between lots of different kernel RFCs :)

Jakub: the issue seems to be that the internals in lib/cpu_rmap.c
are needed to call cpu_rmap_update. It's probably possible to expose
them somehow so that a generic IRQ notifier could call
cpu_rmap_update, as you mentioned, but some rewiring is going to be
needed, I think.

I had a couple ideas for rewiring stuff, but I haven't had time to
context switch back on to this work as I've been busy with a few
other things (the IRQ suspension stuff and another mlx5 thing I have
yet to send upstream).

I hope to take another look at it this week, but I welcome any
suggestions from Shay/Thomas in the meantime.

- Joe