netdev - Re: [PATCH net-next v2] net/smc: Reduce overflow of smc clcsock listen queue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YdVWuWx+ygpD5gjX@TonyMac-Alibaba>
Date:   Wed, 5 Jan 2022 16:28:41 +0800
From:   Tony Lu <tonylu@...ux.alibaba.com>
To:     "D. Wythe" <alibuda@...ux.alibaba.com>
Cc:     Karsten Graul <kgraul@...ux.ibm.com>, kuba@...nel.org,
        davem@...emloft.net, netdev@...r.kernel.org,
        linux-s390@...r.kernel.org, linux-rdma@...r.kernel.org
Subject: Re: [PATCH net-next v2] net/smc: Reduce overflow of smc clcsock
 listen queue

On Wed, Jan 05, 2022 at 12:40:49PM +0800, D. Wythe wrote:
> Hi, 
> 
> Since we are trying to use the backlog parameter to limit smc dangling
> connections, it's seems there's no difference from increasing the
> backlog parameter for the TCP listen socket, user space Application can
> simply avoid the 10K connections problem through that.
> 
> If so, this patch looks redundant to me. Look forward to your advise.
> 
> Thanks.

Hi D. Wythe,

IMHO, this patch is still useful for flood-connections scenes.
Karsten concerns the safety in flood case, we should limit the number
of tcp sockets for safety.

Imaging these two scenes:
- general socket setup process, if with this patch, setting up tcp
  socket is faster, but we also need to wait for the rest part of
  handshake (rdma...). Meanwhile, there are lots of dangling sockets,
  which may cause kernel OOM. If it isn't first-connected peer, the
  rest part is faster, and this patch should be useful.
- always fallback to tcp, the rest part of handshake failed, so we need
  to fallback, this patch should make it faster to fallback.

Whatever scenes we met, it could be better to limit the number of
"half-connected", like backlog in tcp. There is a possible approach,
using sysctl to limit the global or net-namespace max backlog.
Currently, smc doesn't support sysctl yet. I sent an incomplete sysctl
patch before, maybe it could help you. If so, I will fix that patch, and
resent it out.

Thanks,
Tony Lu
 
> On Tue, Jan 04, 2022 at 02:45:35PM +0100, Karsten Graul wrote:
> > On 04/01/2022 14:12, D. Wythe wrote:
> > > From: "D. Wythe" <alibuda@...ux.alibaba.com>
> > > 
> > > In nginx/wrk multithread and 10K connections benchmark, the
> > > backend TCP connection established very slowly, and lots of TCP
> > > connections stay in SYN_SENT state.
> > 
> > I see what you are trying to solve here.
> > So what happens with your patch now is that we are accepting way more connections
> > in advance and queue them up for the SMC connection handshake worker.
> > The connection handshake worker itself will not run faster with this change, so overall
> > it should be the same time that is needed to establish all connections.
> > What you solve is that when 10k connections are started at the same time, some of them
> > will be dropped due to tcp 3-way handshake timeouts. Your patch avoids that but one can now flood
> > the stack with an ~infinite amount of dangling sockets waiting for the SMC handshake, maybe even 
> > causing oom conditions.
> > 
> > What should be respected with such a change would be the backlog parameter for the listen socket,
> > i.e. how many backlog connections are requested by the user space application?
> > There is no such handling of backlog right now, and due to the 'braking' workers we avoided
> > to flood the kernel with too many dangling connections. With your change there should be a way to limit
> > this ind of connections in some way.