netdev - Re: Is bug 200755 in anyone's queue??

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+FuTSf4iLXh-+ADfBNxqcsw=u_vGm7Wsx7vchgwgwvGFYOA6w@mail.gmail.com>
Date:   Fri, 30 Aug 2019 16:30:39 -0400
From:   Willem de Bruijn <willemdebruijn.kernel@...il.com>
To:     Eric Dumazet <eric.dumazet@...il.com>
Cc:     Steve Zabele <zabele@...cast.net>,
        Network Development <netdev@...r.kernel.org>,
        shum@...ndrew.org, vladimir116@...il.com, saifi.khan@...ikr.in,
        Daniel Borkmann <daniel@...earbox.net>, on2k16nm@...il.com,
        Stephen Hemminger <stephen@...workplumber.org>
Subject: Re: Is bug 200755 in anyone's queue??

On Fri, Aug 30, 2019 at 4:54 AM Eric Dumazet <eric.dumazet@...il.com> wrote:
>
>
>
> On 8/29/19 9:26 PM, Willem de Bruijn wrote:
>
> > SO_REUSEPORT was not intended to be used in this way. Opening
> > multiple connected sockets with the same local port.
> >
> > But since the interface allowed connect after joining a group, and
> > that is being used, I guess that point is moot. Still, I'm a bit
> > surprised that it ever worked as described.
> >
> > Also note that the default distribution algorithm is not round robin
> > assignment, but hash based. So multiple consecutive datagrams arriving
> > at the same socket is not unexpected.
> >
> > I suspect that this quick hack might "work". It seemed to on the
> > supplied .c file:
> >
> >                   score = compute_score(sk, net, saddr, sport,
> >                                         daddr, hnum, dif, sdif);
> >                   if (score > badness) {
> >   -                       if (sk->sk_reuseport) {
> >   +                       if (sk->sk_reuseport && !sk->sk_state !=
> > TCP_ESTABLISHED) {

This won't work for a mix of connected and connectionless sockets, of
course (even ignoring the typo), as it only skips reuseport on the
connected sockets.

> >
> > But a more robust approach, that also works on existing kernels, is to
> > swap the default distribution algorithm with a custom BPF based one (
> > SO_ATTACH_REUSEPORT_EBPF).
> >
>
> Yes, I suspect that reuseport could still be used by to load-balance incoming packets
> targetting the same 4-tuple.
>
> So all sockets would have the same score, and we would select the first socket in
> the list (if not applying reuseport hashing)

Can you elaborate a bit?

One option I see is to record in struct sock_reuseport if any port in
the group is connected and, if so, don't return immediately on the
first reuseport_select_sock hit, but continue the search for a higher
scoring connected socket.

Or do return immediately, but do this refined search in
reuseport_select_sock itself, as it has a reference to all sockets in the
group in sock_reuseport->socks[]. Instead of the straightforward hash.

Steve, Re: your point on a scalable QUIC server. That is an
interesting case certainly. Opening a connected socket per flow adds
both memory and port table pressure. I once looked into an SO_TXONLY
udp socket option that does not hash connected sockets into the port
table. In effect receiving on a small set of listening sockets (e.g.,
one per cpu) and sending over separate tx-only sockets. That still
introduces unnecessary memory allocation. OTOH it amortizes some
operations, such as route lookup.

Anyway, that does not fix the immediate issue you reported when using
SO_REUSEPORT as described.