netdev - NOARP devices and NOARP arp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180111210524.GD740@surrealistic.net>
Date:   Thu, 11 Jan 2018 13:05:24 -0800
From:   Jim Westfall <jwestfall@...realistic.net>
To:     netdev@...r.kernel.org
Subject: NOARP devices and NOARP arp_cache entires

Hi

I'm seeing some weird behavior related to NOARP devices and NOARP 
entries in the arp cache.

I have a couple gre tunnels between a linux box and a upstream router that 
send/recv a large amount of packets with unique ips.  On the order of 10k+ 
unique ips per second seen by the linux box.

Each one of the ips ends up getting added to the arp cache as

<ip> dev tun1234 lladdr 0.0.0.0 NOARP

This of course makes the arp cache grow extremely fast and overflow.  
While I can tweak gc_thresh1/2/3 to make arp cache size huge, it doesn't 
seem like the right answer as the kernel is spinning its wheels having to
adding/expunging entries for the high rate of unique ips.

I'm unclear why the kernel is even tracking them in the arp cache.  If 
routing table says to route the packet out a NOARP interface then there is 
no arp, why involve the arp cache at all?  

You can see the behavior with the following

[root@...stfall:~]# uname -a
Linux jwestfall.jwestfall.net 4.14.10_1 #1 SMP PREEMPT Sun Dec 31 20:23:29 UTC 2017 x86_64 GNU/Linux

[root@...stfall:~]# ip neigh show nud noarp 
10.0.0.172 dev lo lladdr 00:00:00:00:00:00 NOARP
10.70.50.5 dev tun0 lladdr 08 NOARP
127.0.0.1 dev lo lladdr 00:00:00:00:00:00 NOARP

Setup a bogus gre tunnel, the remote ip doesn't matter
[root@...stfall:~]# ip tunnel add tun1234 mode gre local 10.0.0.172 remote 10.0.0.156 dev enp4s0
[root@...stfall:~]# ip link set up dev tun1234

Route a bogus network to the tunnel
[root@...stfall:~]# ip route add 192.168.111.0/24 dev tun1234

Ping ips on the bogus network
[root@...stfall:~]# nmap -sP 192.168.111.0/24

Starting Nmap 7.60 ( https://nmap.org ) at 2018-01-11 12:06 PST
...

[root@...stfall:~]# ip neigh show nud noarp 
192.168.111.18 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.4 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.28 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.17 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.14 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.34 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.3 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.20 dev tun1234 lladdr 0.0.0.0 NOARP
10.0.0.172 dev lo lladdr 00:00:00:00:00:00 NOARP
192.168.111.6 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.27 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.13 dev tun1234 lladdr 0.0.0.0 NOARP
192.168.111.33 dev tun1234 lladdr 0.0.0.0 NOARP
...

Also somewhat interesting is that on older kernels (3.2 time range) these 
NOARP entries didn't get added for ipv4, but they did for ipv6 if you 
pushed ipv6 through the ipv4 tunnel.

2804:14c:f281:a1d8:61a2:a30:989f:3eb1 dev tun1 lladdr 0.0.0.0 NOARP
2607:8400:2122:4:e9f9:dbb8:2d44:75d1 dev tun2 lladdr 0.0.0.0 NOARP

Thanks
Jim Westfall