netdev - Re: two gateways with one NIC

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <1176141931.686.120.camel@owl.home.ie>
Date:	Mon, 09 Apr 2007 19:05:31 +0100
From:	W Agtail <wagtail@....ie>
To:	Lennart Sorensen <lsorense@...lub.uwaterloo.ca>
Cc:	Patrick McHardy <kaber@...sh.net>, netdev@...r.kernel.org
Subject: Re: two gateways with one NIC

Nice one, but unfortunately still doesn't work.
I'm now not seeing any marked messages in /var/log/messages and traffic
still going via gw2 for port 8088.

Thanks again. 

On Mon, 2007-04-09 at 13:23 -0400, Lennart Sorensen wrote:
> On Mon, Apr 09, 2007 at 06:02:23PM +0100, W Agtail wrote:
> > Thanks Patrick for your comments too.
> > It seems that you can't mix PREROUTING with --sport or -o.
> > I've also changed the ip rule tables to higher numbers, so I now have:
> 
> I thought you could have --sport, but NOT -o.  No need for -o of course.
> 
> > iptables -t mangle -A PREROUTING -p tcp --dport 8088 -i eth0 -j LOG
> > --log-prefix "fwmark 1: "
> > iptables -t mangle -A PREROUTING -p tcp --dport 8089 -i eth0 -j LOG
> > --log-prefix "fwmark 2: "
> > 
> > iptables -t mangle -A PREROUTING -p tcp --dport 8088 -i eth0 -j MARK
> > --set-mark 1
> > iptables -t mangle -A PREROUTING -p tcp --dport 8089 -i eth0 -j MARK
> > --set-mark 2
> > iptables -t mangle -A PREROUTING -m mark --mark 1 -j LOG --log-prefix
> > "marked 1: "
> > iptables -t mangle -A PREROUTING -m mark --mark 2 -j LOG --log-prefix
> > "marked 2: "
> 
> The thing is that the destination port will NEVER be 8088 for the
> outgoing packets from apache.  The source port will be.
> 
> Try this:
> 
> iptables -t mangle -A PREROUTING -p tcp --sport 8088 -j LOG --log-prefix "fwmark 1: "
> iptables -t mangle -A PREROUTING -p tcp --sport 8089 -j LOG --log-prefix "fwmark 2: "
> iptables -t mangle -A PREROUTING -p tcp --sport 8088 -j MARK --set-mark 1
> iptables -t mangle -A PREROUTING -p tcp --sport 8089 -j MARK --set-mark 2
> iptables -t mangle -A PREROUTING -m mark --mark 1 -j LOG --log-prefix "marked 1: "
> iptables -t mangle -A PREROUTING -m mark --mark 2 -j LOG --log-prefix "marked 2: "
> 
> > ip route add table 8088 default via 10.18.35.11 dev eth0
> > ip route add table 8089 default via 10.18.35.21 dev eth0
> > 
> > ip rule add fwmark 1 table 8088
> > ip rule add fwmark 2 table 8089
> > 
> > 
> > # Confirmation of syntax:
> > iptables -t mangle --list -v -n
> > Chain PREROUTING (policy ACCEPT 5921 packets, 403K bytes)
> >  pkts bytes target     prot opt in     out     source
> > destination
> >    18   984 LOG        tcp  --  eth0   *       0.0.0.0/0
> > 0.0.0.0/0           tcp dpt:8088 LOG flags 0 level 4 prefix `fwmark 1: '
> >     0     0 LOG        tcp  --  eth0   *       0.0.0.0/0
> > 0.0.0.0/0           tcp dpt:8089 LOG flags 0 level 4 prefix `fwmark 2: '
> >    18   984 MARK       tcp  --  eth0   *       0.0.0.0/0
> > 0.0.0.0/0           tcp dpt:8088 MARK set 0x1
> >     0     0 MARK       tcp  --  eth0   *       0.0.0.0/0
> > 0.0.0.0/0           tcp dpt:8089 MARK set 0x2
> >    18   984 LOG        all  --  *      *       0.0.0.0/0
> > 0.0.0.0/0           MARK match 0x1 LOG flags 0 level 4 prefix `marked 1:
> > '
> >     0     0 LOG        all  --  *      *       0.0.0.0/0
> > 0.0.0.0/0           MARK match 0x2 LOG flags 0 level 4 prefix `marked 2:
> > '
> > 
> > 
> > ip rule list
> > 0:      from all lookup local
> > 32764:  from all fwmark 0x2 lookup 8089
> > 32765:  from all fwmark 0x1 lookup 8088
> > 32766:  from all lookup main
> > 32767:  from all lookup default
> > 
> > ip route list table 8088; ip route list table 8089
> > default via 10.18.35.11 dev eth0
> > default via 10.18.35.21 dev eth0
> > 
> > This is what I see in web2's /var/log/messages:
> > Apr  9 06:46:58 web2-fc6 kernel: fwmark 1: IN=eth0 OUT=
> > MAC=00:0c:29:d1:08:48:00:0c:29:49:04:9f:08:00 SRC=192.168.0.241
> > DST=10.18.35.52 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=42359 DF PROTO=TCP
> > SPT=33321 DPT=8088 WINDOW=5840 RES=0x00 SYN URGP=0
> > 
> > Apr  9 06:46:58 web2-fc6 kernel: marked 1: IN=eth0 OUT=
> > MAC=00:0c:29:d1:08:48:00:0c:29:49:04:9f:08:00 SRC=192.168.0.241
> > DST=10.18.35.52 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=42359 DF PROTO=TCP
> > SPT=33321 DPT=8088 WINDOW=5840 RES=0x00 SYN URGP=0
> > 
> > As you can see, packets appear to be marked.
> > But here's a tcpdump on gw2's eth1:
> > 
> > 07:20:35.004205 192.168.0.241.59438 > 10.18.35.52.8088: S
> > 221760494:221760494(0) win 5840 <mss 1460,sackOK,timestamp 1320423
> > 0,nop,wscale 6> (DF)
> > 07:20:35.013144 10.18.35.52.8088 > 192.168.0.241.59438: S
> > 2705868365:2705868365(0) ack 221760495 win 5792 <mss
> > 1460,sackOK,timestamp 2191014 1320423,nop,wscale 1> (DF)
> > 07:20:35.021857 192.168.0.241.59438 > 10.18.35.52.8088: R
> > 221760495:221760495(0) win 0 (DF)
> > 07:20:38.069688 192.168.0.241.59438 > 10.18.35.52.8088: S
> > 221760494:221760494(0) win 5840 <mss 1460,sackOK,timestamp 1321173
> > 0,nop,wscale 6> (DF)
> > 07:20:38.069695 10.18.35.52.8088 > 192.168.0.241.59438: S
> > 2706988830:2706988830(0) ack 221760495 win 5792 <mss
> > 1460,sackOK,timestamp 2192135 1321173,nop,wscale 1> (DF)
> > 07:20:38.071232 192.168.0.241.59438 > 10.18.35.52.8088: R
> > 221760495:221760495(0) win 0 (DF)
> > 
> > So, traffic is being returned via gw2, rather than gw1 :(
> 
> They are marked I guess, but much too late.
> 
> --
> Len Sorensen
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html