Message-ID: <CADvbK_f-WCKp-_NJYOL=j__kxpFuXraFLst3=aPn6BOvX=o+Qg@mail.gmail.com>
Date: Thu, 2 May 2024 14:08:14 -0400
From: Xin Long <lucien.xin@...il.com>
To: Stefan Metzmacher <metze@...ba.org>
Cc: network dev <netdev@...r.kernel.org>, davem@...emloft.net, kuba@...nel.org, 
	Eric Dumazet <edumazet@...gle.com>, Paolo Abeni <pabeni@...hat.com>, 
	Steve French <smfrench@...il.com>, Namjae Jeon <linkinjeon@...nel.org>, 
	Chuck Lever III <chuck.lever@...cle.com>, Jeff Layton <jlayton@...nel.org>, 
	Sabrina Dubroca <sd@...asysnail.net>, Tyler Fanelli <tfanelli@...hat.com>, 
	Pengtao He <hepengtao@...omi.com>, 
	"linux-cifs@...r.kernel.org" <linux-cifs@...r.kernel.org>, 
	Samba Technical <samba-technical@...ts.samba.org>
Subject: Re: [RFC PATCH net-next 0/5] net: In-kernel QUIC implementation with
 Userspace handshake

On Mon, Apr 29, 2024 at 11:20 AM Stefan Metzmacher <metze@...ba.org> wrote:
>
> Hi Xin Long,
>
> >>
> > Just confirmed from other ebpf experts, there are no in-kernel interfaces
> > for loading and interacting with BPF maps/programs(other than from BPF itself).
> >
> > It seems that we have to do this match in QUIC stack. In the latest QUIC
> > code, I added quic_packet_get_alpn(), a 59-line function, to parse ALPNs
> > and then it will search for the listen sock with these ALPNs in
> > quic_sock_lookup().
> >
> > I introduced 'alpn_match' module param, and it can be enabled when loading
> > the module QUIC by:
> >
> >    # modprobe quic alpn_match=1
> >
> > You can test it by tests/sample_test in the latest code:
> >
> >    Start 3 servers:
> >
> >      # ./sample_test server 0.0.0.0 1234 \
> >          ./keys/server-key.pem ./keys/server-cert.pem smbd
> >      # ./sample_test server 0.0.0.0 1234 \
> >          ./keys/server-key.pem ./keys/server-cert.pem h3
> >      # ./sample_test server 0.0.0.0 1234 \
> >          ./keys/server-key.pem ./keys/server-cert.pem ksmbd
> >
> >    Try to connect on clients with:
> >
> >      # ./sample_test client 127.0.0.1 1234 ksmbd
> >      # ./sample_test client 127.0.0.1 1234 smbd
> >      # ./sample_test client 127.0.0.1 1234 h3
> >
> >    to see if the corresponding server responds.
> >
> > There might be some concerns but it's also a useful feature that can not
> > be implemented in userland QUICs. The commit is here:
> >
> > https://github.com/lxin/quic/commit/de82f8135f4e9196b503b4ab5b359d88f2b2097f
> >
> > Please check if this is enough for SMB applications.
>
> It look great thanks!
>
> > Note as a listen socket is now identified by [address + port + ALPN] when
> > alpn_match=1, this feature does NOT require SO_REUSEPORT socket option to
> > be set, unless one wants multiple sockets to listen to
> > the same [address + port + ALPN].
>
> I'd argue that this should be the default and be required before listen()
> or maybe before bind(), so that it can return EADDRINUSE. As EADDRINUSE should only
> happen for servers it might be useful to have a QUIC_SOCKOPT_LISTEN_ALPN instead of
> QUIC_SOCKOPT_ALPN. As QUIC_SOCKOPT_ALPN on a client socket should not generate let
> bind() care about the alpn value at all.
The latest patches have made it always do alpn_match in kernel, and also
support multiple ALPNs(split by ',' when setting it via sockopt) on both
server and client side. Feel free to check.

Note that:
1. As you expected, setsockopt(QUIC_SOCKOPT_ALPN) must be called before
   listen(), and it will return EADDRINUSE if there's a socket already
   listening to the same IP + PORT + ALPN.

2. ALPN bind/match is a *listening* sockets thing, so it checks ALPN only
   when adding listening sockets in quic_hash(), and it does ALPN only
   when looking up listening sockets in quic_sock_lookup().

   By setting ALPNs in client sockets it will ONLY pack these ALPNs into
   the Client Initial Packet when starting connecting, no bind/match for
   these regular sockets, as these sockets can be found by 4-tuple or
   a source_connection_id. bind() doesn't need to care about ALPN for
   client/regular socket either.

   So it's fine to use QUIC_SOCKOPT_ALPN sockopt for both listen and
   regular/client sockets, as in kernel it acts differently on ALPNs
   for listening and regular sockets. (sorry for confusing, I could
   have moved created another hashtable for listening sockets)

   In other word, a listen socket is identified by

        local_ip + local_port + ALPN(s)

   while a regular socket (represents a quic connection) is identified by:

       local_ip + local_port + remote_ip + remote_port

   or any of those

       source_connection_ids.

3. SO_REUSEPORT is still applied to do some load balance between multiple
   processes listening to the same IP + PORT + ALPN, like:

   on server:
   process A: skA = listen(127.0.0.1:1234:smbd)
   process B: skB = listen(127.0.0.1:1234:smbd)
   process C: skC = listen(127.0.0.1:1234:smbd)

   on client:
   connect(127.0.0.1:1234:smbd)
   connect(127.0.0.1:1234:smbd)
   ...

   on server it will select the sk among (skA, skB and skC) based on the
   source address + port in the request from client.

4. Not sure if multiple ALPNs support are useful to you, here is some
   example about how it works:
   - Without SO_REUSEPORT set:

     On server:
     process A: skA = listen(127.0.0.1:1234:smbd,h3,ksmbd)
     process B: skB = listen(127.0.0.1:1234:smbd,h3,ksmbd)

     listen() in process B fails and returns EADDRINUSE.

   - with SO_REUSEPORT set:

     On server:
     process A: skA = listen(127.0.0.1:1234:smbd,h3,ksmbd)
     process B: skB = listen(127.0.0.1:1234:smbd,h3,ksmbd)

     listen() in process B works.

   - with or without SO_REUSEPORT set:

     On server:
     process A: skA = listen(127.0.0.1:1234:h3,ksmbd)
     process B: skB = listen(127.0.0.1:1234:h3,smbd).
     (there's overlap on ALPN list but not exact the same ALPNs)

     listen() in process B fails and returns EADDRINUSE.

   - the match priority for multiple ALPNs is based on the order on the
     client ALPN list:

     On server:
     process A: skA = listen(127.0.0.1:1234:smbd)
     process B: skB = listen(127.0.0.1:1234:h3)
     process C: skC = listen(127.0.0.1:1234:ksmbd)

     On client:
     process X: skX = connect(27.0.0.1:1234:h3,ksmbd,smbd)

     skB will be the one selected to accept the connection, as h3 is the
     1st ALPN on the client ALPN list 'h3,ksmbd,smbd'.

>
> For listens on tcp you also need to specify an explicit port (at least in order
> to be useful).
>
> And it would mean that all application would use it and not block other applications
> from using an explicit alpn.
>
> Also an module parameter for this means the administrator would have to take care
> of it, which means it might be unuseable if loaded with it.
Agree, already dropped this param.

>
> I hope to find some time in the next weeks to play with this.
> Should be relatively trivial create a prototype for samba's smbd.
Sounds Cool!

Thanks.

Powered by blists - more mailing lists