netdev - mlx5en not able to communicate between VF and PF

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:   Thu, 20 Apr 2017 14:06:32 -0600
From:   Robert LeBlanc <robert@...lancnet.us>
To:     netdev@...r.kernel.org
Subject: mlx5en not able to communicate between VF and PF

After figuring out that CentOS requires a newer version of iproute to
enable trust mode on VFs, I'm now having a problem getting a VM on a
bridge on a VF to communicate with the host on the PF or a different
VF.

The set-up:

             Host1                                    Host2

 /--------------------------------------------\
 |                                            |
PF - 192.168.13.13/24                        PF - 192.168.13.14/24
 +-- VF1 - (Move the address
 |          from PF for testing)
 \-- VF2 - br0
            +-- VM1 - 192.168.13.101/24
            +-- VM2 - 192.168.13.102/24
            \-- VM3 - 192.168.13.103/24

Host1 and Host2 have a Mellanox ConnectX-4 100Gb single port adapter
and are connected back-to-back with no switch between them. Host1 VF2
has trust mode set to on. From Host2, I can ping any address on Host1
(PF, VM1, VM2, VM3). From VM3, I can ping VM1, VM2 and Host2 PF. From
Host1 PF, I can ping Host2 PF.

The problem is that none of the VMs can ping Host1 PF nor can the
Host1 PF ping any of the VMs.

While doing tcpdump on the interfaces and a ping from VM3, I can see
the ARP request go through VF2, I can see the request and the response
on PF, but the response never makes it back to VF2.

When I ping from Host1 PF to VM3, I see the ARP request and response
on both the PF and VF2, however the ICMP request is sent on the PF,
but the VF2 never sees it.

If I move the Host1 PF address to a VF, the same problem happens.

It seems to me that some rule in the eswitch is getting in the way,
but I don't know how to view/modify the rules in the eswitch.

Any help getting this working is appreciated.

Thank you,
----------------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1