netdev - Re: [PATCH net-next] net: ip: lookup the best matched listen socket

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAAVpQUD-x1rCZNvPb1nTpzn276gZZKC1DDNxagdiLdpOp=KLHg@mail.gmail.com>
Date: Thu, 31 Jul 2025 10:52:46 -0700
From: Kuniyuki Iwashima <kuniyu@...gle.com>
To: Eric Dumazet <edumazet@...gle.com>
Cc: Menglong Dong <menglong8.dong@...il.com>, ncardwell@...gle.com, davem@...emloft.net, 
	dsahern@...nel.org, kuba@...nel.org, pabeni@...hat.com, horms@...nel.org, 
	netdev@...r.kernel.org, linux-kernel@...r.kernel.org, 
	Martin KaFai Lau <kafai@...com>, Craig Gallek <kraig@...gle.com>
Subject: Re: [PATCH net-next] net: ip: lookup the best matched listen socket

On Thu, Jul 31, 2025 at 6:01 AM Eric Dumazet <edumazet@...gle.com> wrote:
>
> On Thu, Jul 31, 2025 at 5:33 AM Menglong Dong <menglong8.dong@...il.com> wrote:
> >
> > For now, the socket lookup will terminate if the socket is reuse port in
> > inet_lhash2_lookup(), which makes the socket is not the best match.
> >
> > For example, we have socket1 and socket2 both listen on "0.0.0.0:1234",
> > but socket1 bind on "eth0". We create socket1 first, and then socket2.
> > Then, all connections will goto socket2, which is not expected, as socket1
> > has higher priority.
> >
> > This can cause unexpected behavior if TCP MD5 keys is used, as described
> > in Documentation/networking/vrf.rst -> Applications.
> >
> > Therefor, we lookup the best matched socket first, and then do the reuse
> > port logic. This can increase some overhead if there are many reuse port
> > socket :/

This kills O(1) lookup for reuseport...

Another option would be to try hard in __inet_hash() to sort
reuseport groups.


> >
> > Signed-off-by: Menglong Dong <dongml2@...natelecom.cn>
>
> I do not think net-next is open yet ?
>
> It seems this would be net material.
>
> Any way you could provide a test ?

Probably it will look like below and make sure we get
the opposite result:

# python3
>>> from socket import *
>>>
>>> s1 = socket()
>>> s1.setsockopt(SOL_SOCKET, SO_REUSEPORT, 1)
>>> s1.bind(('localhost', 8000))
>>> s1.setsockopt(SOL_SOCKET, SO_BINDTODEVICE, b'lo')
>>> s1.listen()
>>>
>>> s2 = socket()
>>> s2.setsockopt(SOL_SOCKET, SO_REUSEPORT, 1)
>>> s2.bind(('localhost', 8000))
>>> s2.listen()
>>>
>>> cs = []
>>> for i in range(3):
...     c = socket()
...     c.connect(('localhost', 8000))
...     cs.append(c)
...
>>> s1.setblocking(False)
>>> s1.accept()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.12/socket.py", line 295, in accept
    fd, addr = self._accept()
               ^^^^^^^^^^^^^^
BlockingIOError: [Errno 11] Resource temporarily unavailable
>>>
>>> s2.accept()
(<socket.socket fd=15, family=2, type=1, proto=0, laddr=('127.0.0.1',
8000), raddr=('127.0.0.1', 44580)>, ('127.0.0.1', 44580))
>>> s2.accept()
(<socket.socket fd=16, family=2, type=1, proto=0, laddr=('127.0.0.1',
8000), raddr=('127.0.0.1', 44584)>, ('127.0.0.1', 44584))
>>> s2.accept()
(<socket.socket fd=15, family=2, type=1, proto=0, laddr=('127.0.0.1',
8000), raddr=('127.0.0.1', 44588)>, ('127.0.0.1', 44588))



>
> Please CC Martin KaFai Lau <kafai@...com>, as this was added in :
>
> commit 61b7c691c7317529375f90f0a81a331990b1ec1b
> Author: Martin KaFai Lau <kafai@...com>
> Date:   Fri Dec 1 12:52:31 2017 -0800
>
>     inet: Add a 2nd listener hashtable (port+addr)

I think this issue exists from day 1 of reuseport support

commit c125e80b88687b25b321795457309eaaee4bf270
Author: Craig Gallek <kraig@...gle.com>
Date:   Wed Feb 10 16:50:40 2016

    soreuseport: fast reuseport TCP socket selection