netdev - Re: [PATCH v3 net-next 2/4] tcp: bind(addr, 0) remove the SO_REUSEADDR restriction when ephemeral ports are exhausted.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sun, 1 Mar 2020 20:49:49 -0800
From:   Eric Dumazet <eric.dumazet@...il.com>
To:     Kuniyuki Iwashima <kuniyu@...zon.co.jp>, eric.dumazet@...il.com
Cc:     davem@...emloft.net, edumazet@...gle.com, kuni1840@...il.com,
        kuznet@....inr.ac.ru, netdev@...r.kernel.org,
        osa-contribution-log@...zon.com, yoshfuji@...ux-ipv6.org
Subject: Re: [PATCH v3 net-next 2/4] tcp: bind(addr, 0) remove the
 SO_REUSEADDR restriction when ephemeral ports are exhausted.



On 3/1/20 8:31 PM, Kuniyuki Iwashima wrote:
> From:   Eric Dumazet <eric.dumazet@...il.com>
> Date:   Sun, 1 Mar 2020 19:42:25 -0800
>> On 2/29/20 3:35 AM, Kuniyuki Iwashima wrote:
>>> Commit aacd9289af8b82f5fb01bcdd53d0e3406d1333c7 ("tcp: bind() use stronger
>>> condition for bind_conflict") introduced a restriction to forbid to bind
>>> SO_REUSEADDR enabled sockets to the same (addr, port) tuple in order to
>>> assign ports dispersedly so that we can connect to the same remote host.
>>>
>>> The change results in accelerating port depletion so that we fail to bind
>>> sockets to the same local port even if we want to connect to the different
>>> remote hosts.
>>>
>>> You can reproduce this issue by following instructions below.
>>>   1. # sysctl -w net.ipv4.ip_local_port_range="32768 32768"
>>>   2. set SO_REUSEADDR to two sockets.
>>>   3. bind two sockets to (address, 0) and the latter fails.
>>>
>>> Therefore, when ephemeral ports are exhausted, bind(addr, 0) should
>>> fallback to the legacy behaviour to enable the SO_REUSEADDR option and make
>>> it possible to connect to different remote (addr, port) tuples.
>>>
>>> This patch allows us to bind SO_REUSEADDR enabled sockets to the same
>>> (addr, port) only when all ephemeral ports are exhausted.
>>>
>>> The only notable thing is that if all sockets bound to the same port have
>>> both SO_REUSEADDR and SO_REUSEPORT enabled, we can bind sockets to an
>>> ephemeral port and also do listen().
>>>
>>> Fixes: aacd9289af8b ("tcp: bind() use stronger condition for bind_conflict")
>>>
>>> Signed-off-by: Kuniyuki Iwashima <kuniyu@...zon.co.jp>
>>
>> I am unsure about this, since this could double the time taken by this
>> function, which is already very time consuming.
> 
> This patch doubles the time on choosing a port only when all ephemeral ports
> are exhausted, and this fallback behaviour can eventually decreases the time
> on waiting for ports to be released. We cannot know when the ports are
> released, so we may not be able to reuse ports without this patch. This
> patch gives more chace and raises the probability to succeed to bind().
> 
>> We added years ago IP_BIND_ADDRESS_NO_PORT socket option, so that the kernel
>> has more choices at connect() time (instead of bind()) time to choose a source port.
>>
>> This considerably lowers time taken to find an optimal source port, since
>> the kernel has full information (source address, destination address & port)
> 
> I also think this option is usefull, but it does not allow us to reuse
> ports that is reserved by bind(). This is because connect() can reuse ports
> only when their tb->fastresue and tb->fastreuseport is -1. So we still
> cannot fully utilize 4-tuples.

The thing is : We do not want to allow active connections to use a source port
that is used for passive connections.

Many supervisions use dump commands like "ss -t src :40000" to list all connections
for a 'server' listening on port 40000,
or use ethtool to direct all traffic for this port on a particular RSS queue.

Some firewall setups also would need to be changed, since suddenly the port could
be used by unrelated applications.

Note that sharing ports space has been a long standing problem, this is why
we use one netns per job, so that we no longer worry about applications being
hungry.