netdev - Re: vrf and multicast problem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <38ecaaaf-1735-9023-2282-5feead8408b7@gmail.com>
Date:   Thu, 10 Mar 2022 13:03:05 -0700
From:   David Ahern <dsahern@...il.com>
To:     Ben Greear <greearb@...delatech.com>,
        netdev <netdev@...r.kernel.org>
Subject: Re: vrf and multicast problem

On 3/10/22 12:33 PM, Ben Greear wrote:
> On 3/9/22 7:54 PM, David Ahern wrote:
>> On 3/9/22 3:31 PM, Ben Greear wrote:
>>> [resend, sorry...sent to wrong mailing list the first time]
>>>
>>> Hello,
>>>
>>> We recently found a somewhat weird problem, and before I go digging into
>>> the kernel source, I wanted to see if someone had an answer already...
>>>
>>> I am binding (SO_BINDTODEVICE) a socket to an Ethernet port that is in a
>>> VRF with a second
>>> interface.  When I try to send mcast traffic out that eth port,
>>> nothing is
>>> seen on the wire.
>>>
>>> But, if I set up a similar situation with a single network port in
>>> a vrf and send multicast, then it does appear to work as I expected.
>>>
>>> I am not actually trying to do any mcast routing here, I simply want to
>>> send
>>> out mcast frames from a port that resides inside a vrf.
>>>
>>> Any idea what might be the issue?
>>>
>>
>> multicast with VRF works. I am not aware of any known issues
> 
> I set up a more controlled network to do some more testing.  I have eth2
> on 192.168.100.x/24 network, and eth1 on 172.16.0.1/16.
> 
> I bind the mcast transmitter to eth1:
> 
> 193 setsockopt(28, SOL_SOCKET, SO_BINDTODEVICE,
> "eth1\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
> 194 setsockopt(28, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
> 195 bind(28, {sa_family=AF_INET, sin_port=htons(8888),
> sin_addr=inet_addr("0.0.0.0")}, 16) = 0
> 196 fcntl(28, F_GETFL)                      = 0x2 (flags O_RDWR)
> 197 fcntl(28, F_SETFL, O_ACCMODE|O_NONBLOCK) = 0
> 198 setsockopt(28, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
> 199 setsockopt(28, SOL_SOCKET, SO_SNDBUF, [64000], 4) = 0
> 200 setsockopt(28, SOL_SOCKET, SO_RCVBUF, [128000], 4) = 0
> 201 getsockopt(28, SOL_SOCKET, SO_RCVBUF, [256000], [4]) = 0
> 202 getsockopt(28, SOL_SOCKET, SO_SNDBUF, [128000], [4]) = 0
> 203 write(3, "1646940176442:  BtbitsIpEndpoint"..., 69) = 69
> 204 setsockopt(28, SOL_IP, IP_TOS, [0], 4)  = 0
> 205 getsockopt(28, SOL_IP, IP_TOS, [0], [4]) = 0
> 206 setsockopt(28, SOL_SOCKET, SO_PRIORITY, [0], 4) = 0
> 207 getsockopt(28, SOL_SOCKET, SO_PRIORITY, [0], [4]) = 0
> 208 write(3, "1646940176442:  UdpEndpBase.cc 2"..., 148) = 148
> 209 setsockopt(28, SOL_IP, IP_MULTICAST_IF, [16781484], 4) = 0
> 210 setsockopt(28, SOL_IP, IP_MULTICAST_TTL, " ", 1) = 0
> 
> That IP_MULTICAST_IF ioctl should be assigning the IP address of
> eth1.
> 
> But when I sniff, I see the mcast packets going out of eth2:
> 
> [root@...22-63e7 lanforge]# tshark -n -i eth2
> Running as user "root" and group "root". This could be dangerous.
> Capturing on 'eth2'
>     1 0.000000000 192.168.100.28 → 225.5.5.1    LANforge 1514 Seq: 474
>     2 0.060868103 192.168.100.226 → 192.168.100.255 ADwin Config 94
>     3 0.060900503 00:0d:b9:41:6a:90 → ff:ff:ff:ff:ff:ff 0x1111 92
> Ethernet II
>     4 0.209523669 192.168.100.28 → 225.5.5.1    LANforge 1514 Seq: 475
> 
> [root@...22-63e7 lanforge]# ifconfig eth1
> eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 172.16.0.1  netmask 255.255.0.0  broadcast 172.16.255.255
>         inet6 fe80::230:18ff:fe01:63e8  prefixlen 64  scopeid 0x20<link>
>         ether 00:30:18:01:63:e8  txqueuelen 1000  (Ethernet)
>         RX packets 1972669  bytes 409744407 (390.7 MiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 5818525  bytes 7341747933 (6.8 GiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>         device memory 0xdf740000-df75ffff
> 
> [root@...22-63e7 lanforge]# ifconfig eth2
> eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 192.168.100.28  netmask 255.255.255.0  broadcast
> 192.168.100.255
>         inet6 fe80::230:18ff:fe01:63e9  prefixlen 64  scopeid 0x20<link>
>         ether 00:30:18:01:63:e9  txqueuelen 1000  (Ethernet)
>         RX packets 24638831  bytes 8874820766 (8.2 GiB)
>         RX errors 26712  dropped 6596663  overruns 0  frame 16757
>         TX packets 1753211  bytes 370552564 (353.3 MiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>         device memory 0xdf720000-df73ffff
> 
> If I disable VRF and use routing-rules based approach, then it works
> as I expect (mcast frames go out of eth1).
> 
> We tested back to quite-old kernels with same symptom, so I think it is not
> a regression.
> 
> Any suggestions on where to start poking at this in the kernel?
> 

can you reproduce this using namespaces and veth pairs? if so, send me
the script and I will take a look.