netdev - sr-iov and bridges (mlx4)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 19 Sep 2015 17:58:23 -0600
From:	Matthew Monaco <Matthew.Monaco@...orado.EDU>
To:	netdev@...r.kernel.org
Subject: sr-iov and bridges (mlx4)

Hello. I have a Mellanox ConnectX-3 Pro EN (MCX314A-BCCT). I'm only using a
single port so it must provide IP for my host as well as connectivity for VMs.
SR-IOV VFs are working great, my KVM VMs have Ethernet and RDMA.

However, I also want to support virtio VMs. Assuming eth0 is the first port on
my mlx nic, I've tried placing VMs on a bridge with the primary physical
interface, and giving an IP for management to a VF:

br0
|--- eth0
|--- VM
|--- VM
vf0 (IP)
vf1 -> VM
vf2 -> VM
vf3 -> VM

I've tried placing VMs on a bridge with one of the VFs and using the primary
iface for IP.

eth0 (IP)
br0
|--- vf0
|--- VM
|--- VM
vf1 -> VM
vf2 -> VM
vf3 -> VM

And I've also tried using a veth pair to really spread things out:

br0 (IP)
|--- eth0
|--- veth-a
br1       |
|--- veth-b
|--- VM
|--- VM
vf1 -> VM
vf2 -> VM
vf3 -> VM

In all cases, VMs with SR-IOV work fine, IP on the host works fine, outbound
DHCP from the virtio VMs work fine, but inbound frames are not making it back to
the VM.

Is there a know limitation of mixing SR-IOV and bridges in general? Does the
SR-IOV switch specific to the mlx4 hardware not work well with linux bridges? ...?

Thanks!
Matt

Download attachment "signature.asc" of type "application/pgp-signature" (474 bytes)