netdev - Re: ipv6 issues after an DDoS for kernel 4.6.3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1467992314.30694.15.camel@edumazet-glaptop3.roam.corp.google.com>
Date:	Fri, 08 Jul 2016 17:38:34 +0200
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Hannes Frederic Sowa <hannes@...essinduktion.org>
Cc:	Toralf Förster <toralf.foerster@....de>,
	netdev@...r.kernel.org
Subject: Re: ipv6 issues after an DDoS for kernel 4.6.3

On Fri, 2016-07-08 at 11:28 -0400, Hannes Frederic Sowa wrote:
> On 08.07.2016 10:14, Eric Dumazet wrote:
> > On Fri, 2016-07-08 at 15:51 +0200, Toralf Förster wrote:
> >> I do run a 4.6.3 hardened Gentoo kernel at a commodity i7 server. A
> >> DDoS with about 300 MBit/sec over 5 mins resulted an issue for ipv6 at
> >> that system.
> >>
> >> The IPv6 monitoring from my ISP told my that the to be monitored
> >> services (80, 443, 52222) weren't reachable any longer at ipv6 (at
> >> ipv4 there was no issue). Restarting the NIC brought back green lights
> >> for the services at the ipv6 ports too.
> > 
> > Hard to tell without knowing DDOS details, but IPv6 lacks some
> > scalability improvements found in IPv4.
> > 
> > IPv4 no longer has a routing cache, but IPv6 still has one.
> 
> The difference is that routing exceptions are stored in "the" trie
> instead of hash tables in the fib nodes. IPv4 limits that by the size of
> the hash tables, in IPv6 we grow to ipv6/route/max_size, which is pretty
> low.
> 
> Only redirects and mtu updates could potentially increase its size.
> Redirects are limited to the same L2 network, MTU updates must hit the
> socket to be acted upon appropriately. All limited to max_size, so I
> currently see a major problem in the routing code.
> 
> Unfortunately your report has not enough details to pinpoint a specific
> problem in the kernel

Well, the typical DDOS simply use SYN flood, right ? ;)

With IPv4, a server can typically absorb 10 Mpps SYN without major
disruption on linux-4.6

With IPv6, kernel hits the route rwlock quite hard, at less than 2 Mpps

    30.44%  [kernel]  [k] ip6_pol_route.isra.49             
    12.93%  [kernel]  [k] fib6_lookup                       
    12.35%  [kernel]  [k] fib6_get_table                    
    10.36%  [kernel]  [k] _raw_read_lock_bh                 
     8.29%  [kernel]  [k] _raw_read_unlock_bh               
     2.02%  [kernel]  [k] dst_release                       
     1.86%  [kernel]  [k] memcpy_erms       

I guess that switching to plain spinlock could help a bit, before major
surgery.