netdev - When routed to VRF, NF _output

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <f55f7161-7ddc-46d1-844e-0f6e92b06dda@average.org>
Date: Fri, 20 Jun 2025 15:38:35 +0200
From: Eugene Crosser <crosser@...rage.org>
To: netdev@...r.kernel.org
Cc: "netfilter-devel@...r.kernel.org" <netfilter-devel@...r.kernel.org>,
 David Ahern <dsahern@...nel.org>, Nicolas Dichtel
 <nicolas.dichtel@...nd.com>, Florian Westphal <fw@...len.de>,
 Pablo Neira Ayuso <pablo@...filter.org>
Subject: When routed to VRF, NF _output_ hook is run unexpectedly

Hello!

It is possible, and very useful, to implement "two-stage routing" by
installing a route that points to a VRF device:

    ip link add vrfNNN type vrf table NNN
    ...
    ip route add xxxxx/yy dev vrfNNN

however this causes surprising behaviour with relation to netfilter
hooks. Namely, packets taking such path traverse _output_ nftables
chain, with conntracking information reset. So, for example, even
when "notrack" has been set in the prerouting chain, conntrack entries
will still be created. Script attached below demonstrates this behaviour.

So, in order to control conntracking behaviour, it is necessary to
install additional rules in the output chain, despite clearly only
forwarding takes place, logically. Also, because "iif" is not available
in the "output" chain, it is difficult to distinguish such vrf-routed
traffic from true "output" traffic in the nftable rule.

I suppose that if the packet is being processed by vrf because it
followed a route pointing to the vrf interface, output netfilter hook
should not be executed. Possibly(?) a forwarding hook should be run
instead, or none?

Thanks for consideration

Eugene

=====
#!/bin/sh

cleanup() {
	for ns in 1 2 3; do
		ip netns del tns$ns
	done
}

trap cleanup EXIT

for ns in 1 2 3; do
	ip netns add tns$ns
done
ip -n tns2 link add ve21 type veth peer ve12 netns tns1
ip -n tns2 link add ve23 type veth peer ve32 netns tns3

ip -n tns1 link set lo up
ip -n tns1 addr add 172.16.1.1/30 dev ve12
ip -n tns1 link set ve12 up
ip -n tns1 route add default via 172.16.1.2 dev ve12

ip -n tns3 link set lo up
ip -n tns3 addr add 172.16.3.1/30 dev ve32
ip -n tns3 addr add 172.16.9.1/30 dev ve32
ip -n tns3 link set ve32 up
ip -n tns3 route add default via 172.16.3.2 dev ve32

ip -n tns2 link set lo up
ip -n tns2 addr add 172.16.1.2/30 dev ve21
ip -n tns2 link set ve21 up
ip -n tns2 addr add 172.16.3.2/30 dev ve23
ip -n tns2 link set ve23 up

ip -n tns2 link add tvrf1 type vrf table 9999
ip -n tns2 link set tvrf1 up
ip -n tns2 route add 172.16.9.0/24 dev tvrf1
ip -n tns2 route add 172.16.9.0/24 via 172.16.3.1 dev ve23 vrf tvrf1

ip netns exec tns2 nft -f - <<__END__
table inet filter {
	chain rawout {
		type filter hook output priority raw; policy accept;
		counter # notrack  ### NEED THIS ADDITIONAL "notrack"
	}
	chain rawpre {
		type filter hook prerouting priority raw; policy accept;
		counter notrack
	}
	chain forward {
		type filter hook forward priority filter; policy accept;
		ct state established,related counter accept
		counter
	}
}
__END__

ip netns exec tns1 ping -q -W1 -c1 172.16.3.1
ip netns exec tns1 ping -q -W1 -c1 172.16.9.1

ip netns exec tns2 nft list ruleset
ip netns exec tns2 conntrack -L

Download attachment "OpenPGP_signature.asc" of type "application/pgp-signature" (489 bytes)