netdev - Re: ipv6 issues after an DDoS for kernel 4.6.3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <bd45b0ae-341f-09ed-18f9-fdebfcad733f@stressinduktion.org>
Date:	Fri, 8 Jul 2016 11:43:14 -0400
From:	Hannes Frederic Sowa <hannes@...essinduktion.org>
To:	Eric Dumazet <eric.dumazet@...il.com>
Cc:	Toralf Förster <toralf.foerster@....de>,
	netdev@...r.kernel.org
Subject: Re: ipv6 issues after an DDoS for kernel 4.6.3

On 08.07.2016 11:38, Eric Dumazet wrote:
> On Fri, 2016-07-08 at 11:28 -0400, Hannes Frederic Sowa wrote:
>> On 08.07.2016 10:14, Eric Dumazet wrote:
>>> On Fri, 2016-07-08 at 15:51 +0200, Toralf Förster wrote:
>>>> I do run a 4.6.3 hardened Gentoo kernel at a commodity i7 server. A
>>>> DDoS with about 300 MBit/sec over 5 mins resulted an issue for ipv6 at
>>>> that system.
>>>>
>>>> The IPv6 monitoring from my ISP told my that the to be monitored
>>>> services (80, 443, 52222) weren't reachable any longer at ipv6 (at
>>>> ipv4 there was no issue). Restarting the NIC brought back green lights
>>>> for the services at the ipv6 ports too.
>>>
>>> Hard to tell without knowing DDOS details, but IPv6 lacks some
>>> scalability improvements found in IPv4.
>>>
>>> IPv4 no longer has a routing cache, but IPv6 still has one.
>>
>> The difference is that routing exceptions are stored in "the" trie
>> instead of hash tables in the fib nodes. IPv4 limits that by the size of
>> the hash tables, in IPv6 we grow to ipv6/route/max_size, which is pretty
>> low.
>>
>> Only redirects and mtu updates could potentially increase its size.
>> Redirects are limited to the same L2 network, MTU updates must hit the
>> socket to be acted upon appropriately. All limited to max_size, so I
>> currently see a major problem in the routing code.
>>
>> Unfortunately your report has not enough details to pinpoint a specific
>> problem in the kernel
> 
> Well, the typical DDOS simply use SYN flood, right ? ;)

Exactly, I can very well imagine that the stack becomes unresponsive
during DDoS, but after the DDoS I don't see a reason why services should
come up like in IPv4.

> With IPv4, a server can typically absorb 10 Mpps SYN without major
> disruption on linux-4.6
> 
> With IPv6, kernel hits the route rwlock quite hard, at less than 2 Mpps
> 
>     30.44%  [kernel]  [k] ip6_pol_route.isra.49             
>     12.93%  [kernel]  [k] fib6_lookup                       
>     12.35%  [kernel]  [k] fib6_get_table                    
>     10.36%  [kernel]  [k] _raw_read_lock_bh                 
>      8.29%  [kernel]  [k] _raw_read_unlock_bh               
>      2.02%  [kernel]  [k] dst_release                       
>      1.86%  [kernel]  [k] memcpy_erms       
> 
> I guess that switching to plain spinlock could help a bit, before major
> surgery.

Hmm, interesting idea.