| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <E1dPc2U-0001mJ-7A@adsl-kenny.stuart.id.au>
Date: Mon, 26 Jun 2017 16:30:25 +1000
From: Russell Stuart <russell-debian@...art.id.au>
To: netdev@...r.kernel.org
Subject: 4.9.30-2: hairpin NAT doesn't work across bridges
Package: src:linux
Version: 4.9.30-2+deb9u1
Severity: normal
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Configuration:
A box running Debian4.9.0-3-amd64 is acting as a NAT'ing router.
It has a single Ethernet NIC and a wireless NIC servicing the local
LAN. These devices are bridged. Since it has only one wired NIC
it is used to connect to both the LAN and internet via a switch.
This means it must do hairpin NAT over the wired NIC.
internet <--> modem <--> switch <--> LAN
[10.99.99.97/30] ^ [10.91.91.0/24]
| ^
+----------------------------------+ | |
| [10.91.91.1/24] eth0=<--/ v antenna LAN |
| [10.99.99.98/30] br0<---+ | | [10.91.91.0/24] |
| wlan0=<-----/ v
| | +---------------=--+
| ip r a default via 10.99.99.97 | | eth-lan0 |
| iptables -t nat -A POSTROUTING \ | | 10.91.91.129/24 |
| -s 10.91.91.0/24 -j MASQUERADE | | |
+----------------------------------+ | ip r a default \ |
| via 10.91.91.1 |
+------------------+
While wlan0 is the reason for bridge exists in my case it doesn't
have to be a wireless connection. Connecting any two Ethernet
devices to the bridge (so it has to do some work) triggers the
problem.
Problem:
10.91.91.129 can not receive packets from the internet. A packet
arriving from the internet hits eth0, then br0, then is mangled by
iptables nat, and then is supposed to be sent out br0, eth0 again.
The mangled version never makes it out of eth0.
Possible cause:
The bridge is implementing it's "never send a packet out over the
interface it arrived on rule" but it this case it's misapplied the
rule: the packet that is to be sent is not the same packet that
arrived earlier on eth0. It has different source and destination IP
addresses and MAC addresses, and in any case is not being reflected -
it hit the INPUT chain, not the FORWARD chain.
Workarounds:
Set the "hairpin" flag on br0. This works if are to be no loops in
the LAN wiring (which will normally be hidden by STP). If there
are a packet storm will soon ensue, followed in my case by chaos
and panic.
An alternate workaround that mostly works is the use ebtables to
make internet packets bypass the bridge:
ebtables -t broute -A BROUTING -d Multicast -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-dst 10.0.0.0/8 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-dst 172.16.0.0/12 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-dst 169.254.0.0/16 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-dst 192.168.0.0/16 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-src 10.0.0.0/8 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-src 172.16.0.0/12 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-src 169.254.0.0/16 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 --ip-src 192.168.0.0/16 -j ACCEPT
ebtables -t broute -A BROUTING -p IPv4 -j DROP
ebtables -t broute -A BROUTING -p IPv6 --ip6-dst fc00::/fc00:: -j ACCEPT
ebtables -t broute -A BROUTING -p IPv6 --ip6-src fc00::/fc00:: -j ACCEPT
ebtables -t broute -A BROUTING -p IPv6 -j DROP
It only "mostly" works because it fails with OpenVPN. OpenVPN gets
TLS errors if the incoming packets don't go via the bridge.
Reproducing:
Run the attached shell script under Debian on a kernel with the
problem. The shell script sets up the configuration shown in the
diagram above using containers created by systemd-nspawn.
Invoking it using "hairpin-bug.sh bridge" creates the conditions
show in the diagram and produces the following output (spurious
selinux warnings produced by systemd-nspawn have been omitted for
clarity):
PING 10.99.99.90 (10.99.99.90) 56(84) bytes of data.
--- 10.99.99.90 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
The script doesn't need an internet to connection to work as it
"emulates" it. 10.99.99.90 is the one and only address on this
emulated internet.
Invoking it using "hairpin-bug.sh direct" creates the conditions
show in the diagram, with one exception: the eth0 device is not
connected to the br0, and IP addresses assigned to br0 have been
moved to eth0. The output in that case is:
PING 10.99.99.90 (10.99.99.90) 56(84) bytes of data.
64 bytes from 10.99.99.90: icmp_seq=1 ttl=63 time=0.080 ms
--- 10.99.99.90 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.080/0.080/0.080/0.000 ms
This invocation method is mostly a unit test for the script - but
it also proves hairpin NAT does normally work, and points towards
the bridge causing this problem.
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEEZqiOeH6lCkTWvjmorNSfiF5UUm4FAllQqfwACgkQrNSfiF5U
Um7NHg//cBWEd6f6Zhd8tGPg2MWFtoWr4GBp9lD5enpKJmjUXdJUqEoi71lWqn4c
ST6K+5EQ2qeyhhTfh+hEgsVWJH1v33xlk+kcUvB7lL4fzh0q+Z4MyPk7yIPCaZNH
cHLP5ec6jcewHeH2uE456uuO6nZ3qVuOV+c2sAqptYRKyXRmMwf65/2YEjwcoEt3
iLa9bi5b27UsaxYIhWigasxToDTVeWuLy+TzK5Tm2M3oi1JM6u7dVXLu7nMuRCRn
vJBanSnEXLCUnX1QEzBp5yzofxsSPMwZwPPjdBuJ49r//Qji8THpfpyPrhvNtUhn
SsrNcnku9mqTUHR0y0HFhZLBVNJNIBKN21MS5bZiTH8CWatXiO5YXYK8Ruut+09c
Cz9gZ4JTqskk1Tk/qTEGBvqM7rc9q2e8BFCqwOdmcnroFCf67RavGtiYsgQHRYm6
SQs8yjGN/3FlXz+djk4GtUYRvZZwmB1z9zCZ68GTDnJFWvH32pas0nH87WoshpoD
5E+pGywDNHZOsHdlFjBx6oH/42wOrElRTMIZxJ6W/QrJRR85dL4XSZ6EcMa0c39T
2FpGXNevIPMHs79rORmo9V4QmV86+8afqRI/pieZvisC1tSIBERCdPxI/2PtLxtS
+eZ9kVTiSGidq7co9S683WIBmA68KzKECF7Rf4BBdrwjguJyloo=
=EUoF
-----END PGP SIGNATURE-----
View attachment "hairpin-bug.sh" of type "text/plain" (3446 bytes)
Powered by blists - more mailing lists