netdev - policy routing vs dnat replies

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CALCETrWs4NDb_4KzJqnQVAkWHuaUS3Xrguuk2wwZGGKo8TqSLA@mail.gmail.com>
Date:	Wed, 3 Oct 2012 17:44:56 -0700
From:	Andy Lutomirski <luto@...capital.net>
To:	Network Development <netdev@...r.kernel.org>
Subject: policy routing vs dnat replies

I hit an annoying policy routing corner case today.  I have a router
with two WAN interfaces (and no BGP).  I have policy routing set up so
that, if a source address matches either of my public networks, then
outgoing packets use the correct interface.  If neither rule matches
(e.g. the source is 0.0.0.0 for source address selection), then the
default route is whichever one I prefer at the moment.  It looks like
this:

$ ip rule
0:	from all lookup local
32766:	from all lookup main
32767:	from all lookup default
40000:	from <net2> lookup isp2
40001:	from <net1> lookup isp1
40010:	from all lookup real_default

The relevant routes are:

default via <gw1> dev eth0.2  table isp1  src <src1>
default via <gw2> dev eth0.3  table isp2  src <src2>
default via <gw2> dev eth0.3  table real_default  src <src2>  metric 101
default via <gw1> dev eth0.2  table real_default  src <src1>  metric 102

(Yes, this is a bit verbose, but I don't know a more concise way to do this.)

This works nicely: if I specifically bind to one of my public
addresses, the corresponding WAN link is used, and if not or if I'm
coming from a private address, then the metrics determine which link
to use.

DNAT breaks it.  I have a rule:
-A PREROUTING -i eth0.2 -d <ip1> -p tcp --dport <port> -j DNAT --to
<internal host>

<ip1> lives on isp1.  Someone sends a SYN.  It gets routed to the
internal host, and that host sends a SYN/ACK back.  The SYN/ACK has a
source ip that isn't on net1 or net2, so it matches the 'lookup
real_default' rule and gets routed to *gw2*.  iptables rewrites the
source address after the routing decision, and my router sends a
packet with a source address belonging to isp1 to isp2's gateway.  The
packet is then dropped.

Is there any way I can either convince iptables to rewrite the source
address in the prerouting hook or to query the conntrack source
address from the policy rule?  Is there a better solution?  I'm
currently using a somewhat gross combination of MARK and fwmark
matches to work around this problem.  One possibility would be:

Thanks,
Andy

P.S.  Linux 3.2 (at least) appears to have a bug: the SYN/ACK has
ctdir ORIGINAL as seen from the the mangle PREROUTING chain.  I'll
send a real bug report for that if I can reproduce it cleanly on a
newer kernel.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html