Message-ID: <20110114101832.GA3170@tugrik.mns.mnsspb.ru>
Date:	Fri, 14 Jan 2011 13:18:32 +0300
From:	Kirill Smelkov <kirr@....spb.ru>
To:	Patrick McHardy <kaber@...sh.net>
Cc:	davem@...emloft.net, netdev@...r.kernel.org,
	Jonathan Corbet <corbet@....net>,
	Boris Kocherov <bkocherov@...il.com>
Subject: Re: net 00/05: routing based send-to-self implementation

 [ Cc'ing Jonathan Corbet and a friend of mine ]

On Thu, Dec 03, 2009 at 12:25:52PM +0100, Patrick McHardy wrote:
> These patches are yet another attempt at adding "send-to-self" functionality,
> allowing to send packets between two local interfaces over the wire. Unlike
> the approaches I've seen so far, this one is purely routing based.
> Especially the oif classification should also be useful for different setups.
> 
> The patchset consists of three parts:
> 
> - the first three patches add oif classification to fib_rules. This can be
>   used create special routing tables for sockets bound to an interface.
> 
> - the fourth patch changes IPv4 and IPv6 to allow to delete the local rule
>   with priority 0. This allows to re-create it using a lower priority and
>   insert new rules below it to force packets with a local destination out
>   on the wire.
> 
> - the fifth patch adds a devinet sysctl to accept packets with local source
>   addresses in fib_validate_source(). This one unfortunately seems to be
>   necessary, I couldn't come up with a method based purely on adding more
>   routes to fool fib_validate_source() into accepting those packets.
> 
> Usage example:
> 
> # move local routing rule to lower priority
> ip rule add pref 1000 lookup local
> ip rule del pref 0
> 
> # only reply to ARP requests for addresses configured on the device
> echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
> 
> # configure device and force packets of bound sockets out on eth1
> ip address add dev eth1 10.0.0.1/24
> echo 1 > /proc/sys/net/ipv4/conf/eth1/accept_local
> ip link set eth1 up
> ip rule add pref 500 oif eth1 lookup 500
> ip route add default dev eth1 table 500
> 
> # configure device and force packets of bound sockets out on eth2
> ip address add dev eth2 10.0.0.2/24
> echo 1 > /proc/sys/net/ipv4/conf/eth2/accept_local
> ip link set eth2 up
> ip rule add pref 501 oif eth2 lookup 501
> ip route add default dev eth2 table 501
> 
> At this point packets between sockets bound to eth1/eth2 will go over the wire.

Patrick, thanks a lot for doing this!

Just a small follow-up: it is possible to setup such loops without
requiring sockets to be bound to devices. The idea is to setup rules
like

    $ ip rule add pref 100 to <ip-on-eth1> lookup 100
    $ ip route add default dev eth0 table 100

so that on TX, packets go through appropriate interfaces.

And for RX, another rules like

    $ ip rule add pref 10 iif eth0 lookup local

so that packets can be received at all.


I've spent several days to find this, debugging and tracing kernel and
trying various variants on how to do it, so I though I'd better share
the info for poor souls like me :)

For completeness, here is the script which will setup tap0/tap1 loop
through virtual vde_switch.

 ( Jonathan, I though something like this could be useful for LDD4 in
   revised snull not needing to play dirty tricks with IP addresses anymore )


---- 8< (mk-tap-loop.sh) ----
#!/bin/sh -e

# reset interfaces
ip link del tap0 2>/dev/null || :
ip link del tap1 2>/dev/null || :

# create interfaces
vde_tunctl -t tap0
vde_tunctl -t tap1

# assign addresses
ip addr add 192.168.23.10/24 dev tap0
ip addr add 192.168.23.11/24 dev tap1

# put ifs up
ip link set tap0 up
ip link set tap1 up

# lower priority of kernel local table to 500
ip rule del pref   0 lookup local 2>/dev/null || :
ip rule del pref 500 lookup local 2>/dev/null || :
ip rule add pref 500 lookup local

# on rx side handle packets by local table, so we can receive them
echo 1 >/proc/sys/net/ipv4/conf/tap0/accept_local
echo 1 >/proc/sys/net/ipv4/conf/tap1/accept_local
ip rule del pref 10 2>/dev/null || :
ip rule del pref 11 2>/dev/null || :
ip rule add pref 10 iif tap0 lookup local
ip rule add pref 11 iif tap1 lookup local

# tx
ip rule del pref 100 2>/dev/null || :
ip rule del pref 101 2>/dev/null || :
ip rule add pref 100 to 192.168.23.10 lookup 100	# tap0 <- tap1
ip rule add pref 101 to 192.168.23.11 lookup 101	# tap1 <- tap0

ip route flush table 100
ip route flush table 101
ip route add default dev tap1 table 100
ip route add default dev tap0 table 101


# ensure (visually) we've set up it ok

echo
echo " >>> rules:"
ip rule

echo
echo " >>> tap(0|1) routing table:"
#routel | grep '\<tap\(0\|1\)\>'
ip route show table all | grep '\<tap\(0\|1\)\>'

# tx path
echo
echo " >>> checking routing for tx path:"
ip route get 192.168.23.10 connected
ip route get 192.168.23.11 connected

# rx path
echo
echo " >>> checking routing for rx path:"
ip route get from 192.168.23.10 to 192.168.23.11 iif tap1
ip route get from 192.168.23.11 to 192.168.23.10 iif tap0



# start switch and connect switch-tap0 and switch-tap1
echo
echo " >>> ready to start vde_switch and connect wires..."
read
screen sh -c 'screen sh -cx "sleep 4; vde_plug2tap tap0"; screen sh -cx "sleep 4; vde_plug2tap tap1"; sh -cx vde_switch'


# now e.g. ping 192.168.23.11 sends packets to tap0 which are received
# on tap1 and ICMP-ECHO'ed by kernel on tap1 and received on tap0.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists