lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Mon, 28 Sep 2020 08:48:00 +0200
From:   Baptiste Jonglez <baptiste@...sofnetworks.org>
To:     David Ahern <dsahern@...il.com>
Cc:     Alarig Le Lay <alarig@...rdarmor.fr>, netdev@...r.kernel.org,
        jack@...ilfillan.uk, Vincent Bernat <bernat@...ian.org>,
        Oliver <bird-o@...net.de>
Subject: Re: IPv6 regression introduced by commit
 3b6761d18bc11f2af2a6fc494e9026d39593f22c

On 27-09-20, David Ahern wrote:
> On 9/27/20 9:10 AM, Baptiste Jonglez wrote:
> > On 27-09-20, Baptiste Jonglez wrote:
> >> 1) failing IPv6 neighbours, what Alarig reported.  We are seeing this
> >>    on a full-view BGP router with rather low amount of IPv6 traffic
> >>    (around 10-20 Mbps)
> > 
> > Ok, I found a quick way to reproduce this issue:
> > 
> >     # for net in {1..9999}; do ip -6 route add 2001:db8:ffff:${net}::/64 via fe80::4242 dev lo; done
> > 
> > and then:
> > 
> >     # for net in {1..9999}; do ping -c1 2001:db8:ffff:${net}::1; done
> > 
> > This quickly gets to a situation where ping fails early with:
> > 
> >     ping: connect: Network is unreachable
> > 
> > At this point, IPv6 connectivity is broken.  The kernel is no longer
> > replying to IPv6 neighbor solicitation from other hosts on local
> > networks.
> > 
> > When this happens, the "fib_rt_alloc" field from /proc/net/rt6_stats
> > is roughly equal to net.ipv6.route.max_size (a bit more in my tests).
> > 
> > Interestingly, the system appears to stay in this broken state
> > indefinitely, even without trying to send new IPv6 traffic.  The
> > fib_rt_alloc statistics does not decrease.
> > 
> 
> fib_rt_alloc is incremented by calls to ip6_dst_alloc. Each of your
> 9,999 pings is to a unique address and hence causes a dst to be
> allocated and the counter to be incremented. It is never decremented.
> That is standard operating procedure.

Ok, then this is a change in behaviour.  Here is a graph of fib_rt_alloc
on a busy router (IPv6 full view, moderate IPv6 traffic) with 4.9 kernel:

  https://files.polyno.me/tmp/rt6_stats_fib_rt_alloc_4.9.png

It varies quite a lot and stays around 50, so clearly it can be
decremented in regular operation.

On 4.19 and later, it does seem to be decremented only when a route is
removed (ip -6 route delete).  Here is the same graph on a router with a
4.19 kernel and a large net.ipv6.route.max_size:

   https://files.polyno.me/tmp/rt6_stats_fib_rt_alloc_4.19.png

Overall, do you mean that fib_rt_alloc is a red herring and is not a good
marker of the issue?

Thanks,
Baptiste

Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)

Powered by blists - more mailing lists