netdev - Re: SR-IOV + switchdev + vlan + Mellanox: Cannot ping

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ZjKHCTe9j4tAg7yp@f4>
Date: Wed, 1 May 2024 14:16:41 -0400
From: Benjamin Poirier <benjamin.poirier@...il.com>
To: Shane Miller <gshanemiller6@...il.com>
Cc: Jiri Pirko <jiri@...nulli.us>, netdev@...r.kernel.org
Subject: Re: SR-IOV + switchdev + vlan + Mellanox: Cannot ping

On 2024-04-30 17:29 -0400, Shane Miller wrote:
> On Mon, Apr 29, 2024 at 7:29 AM Jiri Pirko <jiri@...nulli.us> wrote:
> > Nope. Think of it as another switch inside the NIC that connects VFs and
> > uplink port. You have representors that represent the switch port. Each
> > representor has counter part VF. You have to configure the forwarding
> > between the representor, similar to switch ports. In switch, there is
> > also no default forwarding.
> 
> The salient phrase is "forward between the representor". You seem to
> be saying to forward ARP packets from the uplink port (ieth3 e.g.
> the NIC that was virtualized) to a port representer (ieth3r0)? Are those
> the correct endpoints?
> 
> Second, what UNIX tool do I use to forward? As far as I can tell, the
> correct methodology is to first create a bridge:
> 
>     ip link add name br0 type bridge
>     ip link set br0 up
> 

I recently learned about this too and here is what I noted down:

In switchdev mode, two netdevs are created for each VF:
1) port representor (PR)
	`ethtool -i` shows "driver: mlx5e_rep"
	sysfs device/ is the PF
	`devlink port` shows "flavour pcivf"
2) actual VF
	driver: mlx5_core
	sysfs device/ is unique
	`devlink port` shows "flavour virtual"

In order to be able to pass traffic, the PR must be added into a bridge
with the PF:
ip link add br0 up type bridge
ip link set dev eth2 up master br0  # PF
ip link set dev eth4 up master br0  # PR