netdev - Re: [PATCH v3 net-next 2/4] tcp: bind(addr, 0) remove the SO_REUSEADDR restriction when ephemeral ports are exhausted.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200302043116.19101-1-kuniyu@amazon.co.jp>
Date:   Mon, 2 Mar 2020 13:31:16 +0900
From:   Kuniyuki Iwashima <kuniyu@...zon.co.jp>
To:     <eric.dumazet@...il.com>
CC:     <davem@...emloft.net>, <edumazet@...gle.com>, <kuni1840@...il.com>,
        <kuniyu@...zon.co.jp>, <kuznet@....inr.ac.ru>,
        <netdev@...r.kernel.org>, <osa-contribution-log@...zon.com>,
        <yoshfuji@...ux-ipv6.org>
Subject: Re: [PATCH v3 net-next 2/4] tcp: bind(addr, 0) remove the SO_REUSEADDR restriction when ephemeral ports are exhausted.

From:   Eric Dumazet <eric.dumazet@...il.com>
Date:   Sun, 1 Mar 2020 19:42:25 -0800
> On 2/29/20 3:35 AM, Kuniyuki Iwashima wrote:
> > Commit aacd9289af8b82f5fb01bcdd53d0e3406d1333c7 ("tcp: bind() use stronger
> > condition for bind_conflict") introduced a restriction to forbid to bind
> > SO_REUSEADDR enabled sockets to the same (addr, port) tuple in order to
> > assign ports dispersedly so that we can connect to the same remote host.
> > 
> > The change results in accelerating port depletion so that we fail to bind
> > sockets to the same local port even if we want to connect to the different
> > remote hosts.
> > 
> > You can reproduce this issue by following instructions below.
> >   1. # sysctl -w net.ipv4.ip_local_port_range="32768 32768"
> >   2. set SO_REUSEADDR to two sockets.
> >   3. bind two sockets to (address, 0) and the latter fails.
> > 
> > Therefore, when ephemeral ports are exhausted, bind(addr, 0) should
> > fallback to the legacy behaviour to enable the SO_REUSEADDR option and make
> > it possible to connect to different remote (addr, port) tuples.
> > 
> > This patch allows us to bind SO_REUSEADDR enabled sockets to the same
> > (addr, port) only when all ephemeral ports are exhausted.
> > 
> > The only notable thing is that if all sockets bound to the same port have
> > both SO_REUSEADDR and SO_REUSEPORT enabled, we can bind sockets to an
> > ephemeral port and also do listen().
> > 
> > Fixes: aacd9289af8b ("tcp: bind() use stronger condition for bind_conflict")
> > 
> > Signed-off-by: Kuniyuki Iwashima <kuniyu@...zon.co.jp>
> 
> I am unsure about this, since this could double the time taken by this
> function, which is already very time consuming.

This patch doubles the time on choosing a port only when all ephemeral ports
are exhausted, and this fallback behaviour can eventually decreases the time
on waiting for ports to be released. We cannot know when the ports are
released, so we may not be able to reuse ports without this patch. This
patch gives more chace and raises the probability to succeed to bind().

> We added years ago IP_BIND_ADDRESS_NO_PORT socket option, so that the kernel
> has more choices at connect() time (instead of bind()) time to choose a source port.
>
> This considerably lowers time taken to find an optimal source port, since
> the kernel has full information (source address, destination address & port)

I also think this option is usefull, but it does not allow us to reuse
ports that is reserved by bind(). This is because connect() can reuse ports
only when their tb->fastresue and tb->fastreuseport is -1. So we still
cannot fully utilize 4-tuples.

https://lore.kernel.org/netdev/20200221100155.76241-1-kuniyu@amazon.co.jp/#t

Thanks.