[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <681384df8e0f1_35e23e294ea@willemb.c.googlers.com.notmuch>
Date: Thu, 01 May 2025 10:27:43 -0400
From: Willem de Bruijn <willemdebruijn.kernel@...il.com>
To: Kuniyuki Iwashima <kuniyu@...zon.com>,
jiayuan.chen@...ux.dev
Cc: davem@...emloft.net,
dsahern@...nel.org,
edumazet@...gle.com,
horms@...nel.org,
kuba@...nel.org,
kuniyu@...zon.com,
linux-kernel@...r.kernel.org,
linux-kselftest@...r.kernel.org,
netdev@...r.kernel.org,
pabeni@...hat.com,
shuah@...nel.org,
willemdebruijn.kernel@...il.com
Subject: Re: [RFC net-next v1 1/2] udp: Introduce UDP_STOP_RCV option for UDP
Kuniyuki Iwashima wrote:
> From: "Jiayuan Chen" <jiayuan.chen@...ux.dev>
> Date: Thu, 01 May 2025 06:22:17 +0000
> > 2025/5/1 12:42, "Kuniyuki Iwashima" <kuniyu@...zon.com> wrote:
> >
> > >
> > > From: Jiayuan Chen <jiayuan.chen@...ux.dev>
> > >
> > > Date: Thu, 1 May 2025 11:51:08 +0800
> > >
> > > >
> > > > For some services we are using "established-over-unconnected" model.
> > > >
> > > >
> > > >
> > > > '''
> > > >
> > > > // create unconnected socket and 'listen()'
> > > >
> > > > srv_fd = socket(AF_INET, SOCK_DGRAM)
> > > >
> > > > setsockopt(srv_fd, SO_REUSEPORT)
> > > >
> > > > bind(srv_fd, SERVER_ADDR, SERVER_PORT)
> > > >
> > > >
> > > >
> > > > // 'accept()'
> > > >
> > > > data, client_addr = recvmsg(srv_fd)
> > > >
> > > >
> > > >
> > > > // create a connected socket for this request
> > > >
> > > > cli_fd = socket(AF_INET, SOCK_DGRAM)
> > > >
> > > > setsockopt(cli_fd, SO_REUSEPORT)
> > > >
> > > > bind(cli_fd, SERVER_ADDR, SERVER_PORT)
> > > >
> > > > connect(cli, client_addr)
> > > >
> > > > ...
> > > >
> > > > // do handshake with cli_fd
> > > >
> > > > '''
> > > >
> > > >
> > > >
> > > > This programming pattern simulates accept() using UDP, creating a new
> > > >
> > > > socket for each client request. The server can then use separate sockets
> > > >
> > > > to handle client requests, avoiding the need to use a single UDP socket
> > > >
> > > > for I/O transmission.
> > > >
> > > >
> > > >
> > > > But there is a race condition between the bind() and connect() of the
> > > >
> > > > connected socket:
> > > >
> > > > We might receive unexpected packets belonging to the unconnected socket
> > > >
> > > > before connect() is executed, which is not what we need.
> > > >
> > > > (Of course, before connect(), the unconnected socket will also receive
> > > >
> > > > packets from the connected socket, which is easily resolved because
> > > >
> > > > upper-layer protocols typically require explicit boundaries, and we
> > > >
> > > > receive a complete packet before creating a connected socket.)
> > > >
> > > >
> > > >
> > > > Before this patch, the connected socket had to filter requests at recvmsg
> > > >
> > > > time, acting as a dispatcher to some extent. With this patch, we can
> > > >
> > > > consider the bind and connect operations to be atomic.
> > > >
> > >
> > > SO_ATTACH_REUSEPORT_EBPF is what you want.
> > >
> > > The socket won't receive any packets until the socket is added to
> > >
> > > the BPF map.
> > >
> > > No need to reinvent a subset of BPF functionalities.
> > >
> >
> > I think this feature is for selecting one socket, not filtering out certain
> > sockets.
> >
> > Does this mean that I need to first capture all sockets bound to the same
> > port, and then if the kernel selects a socket that I don't want to receive
> > packets on, I'll need to implement an algorithm in the BPF program to
> > choose another socket from the ones I've captured, in order to avoid
> > returning that socket?
>
> Right.
>
> If you want a set of sockets to listen on the port, you can implement
> as such with BPF; register the sockets to the BPF map, and if kernel pick
> up other sockets and triggers the BPF prog, just return one of the
> registerd sk.
>
> Even when you have connect()ed sockets on the same port, kernel will
> fall back to the normal scoring to find the best one, and it's not a
> problem as the last 'result' is one selected by BPF or a connected sk,
> and the packet won't be routed to not-yet-registered unconnected sk.
>
>
> >
> > This looks like it completely bypasses the kernel's built-in scoring
> > logic. Or is expanding BPF_PROG_TYPE_SK_REUSEPORT to have filtering
> > capabilities also an acceptable solution?
Reuseport BPF exists because we want to avoid having to continue to
add custom rules in C for each scenario.
In this case, I did wonder whether it is possible to avoid hitting
the soon-to-be connected socket with the standard reuseport
algorithm in reuseport_select_sock_by_hash.
Setting SO_INCOMING_CPU to a cpu on which no packets arrive will
lower its priority relative to other sockets. It's a bit of a hack,
but should work?
Powered by blists - more mailing lists