netdev - Re: [PATCH v1] io_uring zcrx: allow sharing of ifqs with other instances

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <e3c4b0c5-72e1-4a2d-a9bf-2e57b1e191ae@kernel.dk>
Date: Fri, 24 Oct 2025 07:01:15 -0600
From: Jens Axboe <axboe@...nel.dk>
To: David Wei <dw@...idwei.uk>, io-uring@...r.kernel.org,
 netdev@...r.kernel.org
Cc: Pavel Begunkov <asml.silence@...il.com>
Subject: Re: [PATCH v1] io_uring zcrx: allow sharing of ifqs with other
 instances

On 10/23/25 3:39 PM, David Wei wrote:
> Each ifq is bound to a HW RX queue with no way to share this across
> multiple io_uring instances. It is possible that one io_uring instance
> will not be able to fully saturate an entire HW RX queue. To handle more
> work the only way is to add additional io_uring instances w/ ifqs, but
> HW RX queues are a limited resource on a system.
> 
> From userspace it is possible to move work from this io_uring instance
> w/ an ifq to other threads, but this will incur context switch overhead.
> What I'd like to do is share an ifq (and hence a HW RX queue) across
> multiple rings.
> 
> Add a way for io_uring instances to clone an ifq from another. This is
> done by passing a new flag IORING_ZCRX_IFQ_REG_CLONE in the registration
> struct io_uring_zcrx_ifq_reg, alongside the fd and ifq id of the ifq to
> be cloned.
> 
> The cloned ifq holds two refs:
>   1. On the source io_ring_ctx percpu_ref
>   2. On the source ifq refcount_t
> 
> This ensures that the source ifq and ring ctx remains valid while there
> are proxies.
> 
> The only way to destroy an ifq today is to destroy the entire ring, so
> both the real ifq and the proxy ifq are freed together.
> 
> At runtime, io_zcrx_recv_frag checks the ifq in the net_iov->priv field.
> This is expected to be the primary ifq that is bound to a HW RX queue,
> and is what prevents another ring from issuing io_recvzc on a zero copy
> socket. Once a secondary ring clones the ifq, this check will pass.
> 
> It's expected for userspace to coordinate the sharing and
> synchronisation of the refill queue when returning buffers. The kernel
> is not involved at all.
> 
> It's also expected userspace to distributed accepted sockets with
> connections steered to zero copy queues across multiple rings for load
> balancing.

I think this would be a lot easier to review, if you split out a few
things. Like:

The locking of two rings, just make that a prep patch. You also need
to change the name, it's too generic. That was fine when it was a
static in a single file, but should be better now.

Add the reference counting. It looks a bit suspicious on the
io_shutdown_zcrx_ifqs() side, having it separate would also make
that clearer.

And then you can have the meat of it on top of those.

Side note - you need to check the return value of kzalloc(), it can
indeed return NULL.

-- 
Jens Axboe