netdev - Re: fwmark based routing stopped working in 2.6.32

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <4B62E0D3.8040207@trash.net>
Date:	Fri, 29 Jan 2010 14:21:23 +0100
From:	Patrick McHardy <kaber@...sh.net>
To:	Nebojsa Trpkovic <nebojsa@...kovic.com>
CC:	linux-net@...r.kernel.org,
	Linux Netdev List <netdev@...r.kernel.org>
Subject: Re: fwmark based routing stopped working in 2.6.32

Nebojsa Trpkovic wrote:
> hello.
> 
> I have two ADSL links on eth2 and eth3.
> 
> ADSL1 (eth2) with IP 10.5.18.18 is default gateway in main routing table.
> 
> ADSL2 (eth3) with IP 10.5.18.22 is used just for marked packets:
> ###################################################
> #!/bin/bash
> ip route add default via 10.5.18.22 dev eth3 table 20
> ip rule add fwmark 0x351 table 20
> ip rule add fwmark 0x352 table 20
> ip rule add fwmark 0x353 table 20
> ip route flush cache
> ###################################################
> 
> everything worked fine for years using kernels 2.6.24 and 2.6.29.
> recently I upgraded to 2.6.32-r2 and traffic through ADSL2 stopped.
> 
> the moment I delete table 20 and ip rules, everything works fine:
> I can set both ADSL1 or ADSL2 as default gateway and they will work.
> 
> again, the moment I start making routing decision considering firewall
> marks, I get traffic only on ADSL1 (main table default gw) interface.
> 
> I've found out that when I mark ICMP protocol with 0x351 fwmark and try
> too ping something, ping packets are sent via eth3 indeed:
> iptraf detailed eth3 statistics shows that there are constatnly outgoing
> ICMP packages.
> 
> even more interesting is fact that there is exactly the same number of
> incoming ICMP packages, but my ping output is empty:
> there is no "Destination Host Unreachable" or similar - nothing.
> 
> this leeds me to believe that ICMP packages are routed right, I receive
> some answer, but those answer packages are discarded.
> 
> so, I've flushed all firewall rules except marking for ICMP, and added
> explicit
> ###################################################
> iptables -t mangle -A OUTPUT -p ICMP -j MARK --set-mark 0x351
> ###################################################
> that didn't help.
> 
> I've added explicit rule
> ###################################################
> iptables -I INPUT -i eth3 -j ACCEPT
> ###################################################
> that didn't help.
> 
> I've checked, and my source route verification is turned off for these
> ifaces:
> ###################################################
> etc # sysctl net.ipv4.conf.default.rp_filter
> net.ipv4.conf.default.rp_filter = 1
> etc # sysctl net.ipv4.conf.eth2.rp_filter
> net.ipv4.conf.eth2.rp_filter = 0
> etc # sysctl net.ipv4.conf.eth3.rp_filter
> net.ipv4.conf.eth3.rp_filter = 0
> ###################################################
> changing that to "=1" doesn't solve the problem.
> 
> tcpdump on eth3 after 3 pings to 216.239.34.10
> ###################################################
> ping -I eth3 -c3 216.239.34.10
> PING 216.239.34.10 (216.239.34.10) from 10.5.18.21 eth3: 56(84) bytes of
> data.
> 
> --- 216.239.34.10 ping statistics ---
> 3 packets transmitted, 0 received, 100% packet loss, time 2006ms
> ###################################################
> ###################################################
> 13:24:23.556436 00:23:54:07:e9:6a > 00:90:d0:da:d2:06, ethertype IPv4
> (0x0800), length 98: 10.5.18.21 > 216.239.34.10: ICMP echo request, id
> 51300, seq 1, length 64
> 13:24:23.605304 00:90:d0:da:d2:06 > 00:23:54:07:e9:6a, ethertype IPv4
> (0x0800), length 98: 216.239.34.10 > 10.5.18.21: ICMP echo reply, id
> 51300, seq 1, length 64
> 13:24:24.555536 00:23:54:07:e9:6a > 00:90:d0:da:d2:06, ethertype IPv4
> (0x0800), length 98: 10.5.18.21 > 216.239.34.10: ICMP echo request, id
> 51300, seq 2, length 64
> 13:24:24.603520 00:90:d0:da:d2:06 > 00:23:54:07:e9:6a, ethertype IPv4
> (0x0800), length 98: 216.239.34.10 > 10.5.18.21: ICMP echo reply, id
> 51300, seq 2, length 64
> 13:24:25.563105 00:23:54:07:e9:6a > 00:90:d0:da:d2:06, ethertype IPv4
> (0x0800), length 98: 10.5.18.21 > 216.239.34.10: ICMP echo request, id
> 51300, seq 3, length 64
> 13:24:25.610497 00:90:d0:da:d2:06 > 00:23:54:07:e9:6a, ethertype IPv4
> (0x0800), length 98: 216.239.34.10 > 10.5.18.21: ICMP echo reply, id
> 51300, seq 3, length 64
> ###################################################
> 
> so, I'm definitely getting those packets back, but system ignoress them.
> 
> any idea what could go wrong and why does my system discard packages
> from eth3 if they are not routed by main ruting table?
> 
> any info on what could be changed between kernels 2.6.29 and 2.6.32
> regarding this issue?

Please try this patch. It might need a few minor changes to apply
cleanly.

View attachment "01.diff" of type "text/x-patch" (2871 bytes)