Date:	Tue, 16 Sep 2008 22:45:14 +0200
From:	"Julius Volz" <juliusv@...gle.com>
To:	"Simon Horman" <horms@...ge.net.au>
Cc:	"Joseph Mack NA3T" <jmack@...d.net>, lvs-devel@...r.kernel.org,
	netdev@...r.kernel.org, j.stubbs@...kthink.co.jp,
	"Siim Põder" <siim@...rad-teel.net>,
	"Vince Busam" <vbusam@...gle.com>
Subject: Re: Adding SNAT support to LVS/NAT

On Mon, Sep 15, 2008 at 3:43 AM, Simon Horman <horms@...ge.net.au> wrote:
> On Sun, Sep 14, 2008 at 04:47:51PM +0200, Julius Volz wrote:
>> On Sun, Sep 14, 2008 at 12:39 PM, Julius Volz <juliusv@...gle.com> wrote:
>> > On Sun, Sep 14, 2008 at 3:37 AM, Joseph Mack NA3T <jmack@...d.net> wrote:
>> >> On Sun, 14 Sep 2008, Julius Volz wrote:
>> >>
>> >>> So maybe it would already work? ;)
>> >>
>> >> No. Some highly motivated people tried doing SNAT on OUTPUT in an attempt to
>> >> do F5-SNAT and it didn't work. This lead to the write up in
>> >>
>> >> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat
>> >>
>> >> which brings us to where we are now.
>> >
>> > Thanks for the info! Right, I even said myself in the previous reply
>> > that ip_vs_postrouting() stops further processing in the POSTROUTING
>> > chain, so it never reaches netfilter NAT code.
>>
>> Actually, what if we modify or remove that function to allow further
>> processing in POSTROUTING? Could SNAT work with IPVS then?
>>
>> The comment above it says that the function specifically wants to
>> avoid further NAT by netfilter. But is this always a problem?
>
> Well, it would be a problem if it gets DNATed a second time.
> But perhaps we can take a slightly different approach such that
> we protect against DNAT while allowing SNAT.

So I did some experiments and removed the ip_vs_post_routing()
function that stops further POSTROUTING processing. I added an SNAT
rule to POSTROUTING like this:

$ iptables –t nat –A POSTROUTING –o eth1 –j SNAT –-to $DIP

Amazingly, the first SYN and the SYN/ACK of a TCP connection to the
VIP:vport do not traverse the NAT chain in POSTROUTING at all
(verified by LOG target), so their source is not rewritten (like in
normal LVS/NAT). However, the client's ACK response to the SYN/ACK
finally appears in this table and only this third packet is SNATed
correctly (the real server then answers with an RST, since it doesn't
know a connection from that source). The packet trace on the director
looks something like this:

Packet 1:
CIP => VIP SYN (eth0, before IPVS)
CIP => RIP SYN (eth1, after IPVS)

Packet 2:
RIP => CIP SYN/ACK (eth1, before IPVS)
VIP => CIP SYN/ACK (eth0, after IPVS)

Packet 3:
CIP => VIP ACK (eth0, before IPVS)
DIP => RIP ACK (eth1, after IPVS) - this is the first successfully
SNATed packet!

Packet 4:
RIP => DIP RST - real server doesn't know this connection from the DIP

Any ideas why the first packets never appear in the POSTROUTING chain?
I do not understand this, since IPVS injects all handled packets into
the OUTPUT chain, whereafter they should traverse POSTROUTING...

Without any IPVS, SNAT works without any problems and as expected,
only the first (SYN) packet of a connection appears on the POSTROUTING
chain.

> Perhaps it might just be easier to allow iptables to explictly match
> packets that have been mangled by LVS-NAT. Perhaps by poviding
> a match rule for skb->ipvs_property? Or by using Siim Põder's match
> against connections in the LVS connection table.
>
> http://lists.graemef.net/pipermail/lvs-users/2008-July/021081.html

Thanks, interesting! Though for processing an already IPVS-ed packet
in POSTROUTING, it should be enough to match on skb->ipvs_property, I
think.

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists