Message-ID: <CAD2tU16x1aeZLcQrESroqz-5n=S0nkgh8QTQO31-yYF_7hqB=Q@mail.gmail.com>
Date: Thu, 11 Jul 2024 10:08:18 +0100
From: Leonardo Milleri <lmilleri@...hat.com>
To: Cindy Lu <lulu@...hat.com>, "Michael S. Tsirkin" <mst@...hat.com>, Parav Pandit <parav@...dia.com>
Cc: Jason Wang <jasowang@...hat.com>, Dragos Tatulea <dtatulea@...dia.com>, 
	"sgarzare@...hat.com" <sgarzare@...hat.com>, "netdev@...r.kernel.org" <netdev@...r.kernel.org>, 
	"virtualization@...ts.linux-foundation.org" <virtualization@...ts.linux-foundation.org>, 
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>, "kvm@...r.kernel.org" <kvm@...r.kernel.org>
Subject: Re: [PATCH v3 0/2] vdpa: support set mac address from vdpa tool

Sorry for the noise, resending the email in text format

Hi All,

My answers inline below

>> Any specific reason to pre-create those large number of vdpa devices of the pool?
>> I was hoping to create vdpa device with needed attributes, when spawning a kubevirt instance.
>> K8s DRA infrastructure [1] can be used to create the needed vdpa device. Have you considered using the DRA of [1]?

The vhost-vdpa devices are created in the host before spawning the
kubevirt VM. This is achieved by using:
- sriov-network-operator: load kernel drivers, create vdpa devices
(with MAC address), etc
- sriov-device-plugin: create pool of resources (vdpa devices in this
case), advertise devices to k8s, allocate devices during pod creation
(by the way, isn't this mechanism very similar to DRA?)

Then we create the kubevirt VM by defining an interface with the
following attributes:
- type:vdpa
- mac
- source: vhost-vdpa path

So the issue is, how to make sure the mac in the VM is the same mac of vdpa?
Two options:
- ensure kubevirt interface mac is equal to vdpa mac: this is not
possible because of the device plugin resource pool. You can have a
few devices in the pool and the device plugin picks one randomly.
- change vdpa mac address at a later stage, to make sure it is aligned
with kubevirt interface mac. I don't know if there is already specific
code in kubevirt to do that or need to be implemented.

Hope this helps to clarify

Thanks
Leonardo


On Wed, Jul 10, 2024 at 10:46 AM Cindy Lu <lulu@...hat.com> wrote:
>
> On Wed, 10 Jul 2024 at 14:10, Michael S. Tsirkin <mst@...hat.com> wrote:
> >
> > On Wed, Jul 10, 2024 at 11:05:48AM +0800, Jason Wang wrote:
> > > On Tue, Jul 9, 2024 at 8:42 PM Michael S. Tsirkin <mst@...hat.com> wrote:
> > > >
> > > > On Tue, Jul 09, 2024 at 02:19:19PM +0800, Cindy Lu wrote:
> > > > > On Tue, 9 Jul 2024 at 11:59, Parav Pandit <parav@...dia.com> wrote:
> > > > > >
> > > > > > Hi Cindy,
> > > > > >
> > > > > > > From: Cindy Lu <lulu@...hat.com>
> > > > > > > Sent: Monday, July 8, 2024 12:17 PM
> > > > > > >
> > > > > > > Add support for setting the MAC address using the VDPA tool.
> > > > > > > This feature will allow setting the MAC address using the VDPA tool.
> > > > > > > For example, in vdpa_sim_net, the implementation sets the MAC address to
> > > > > > > the config space. However, for other drivers, they can implement their own
> > > > > > > function, not limited to the config space.
> > > > > > >
> > > > > > > Changelog v2
> > > > > > >  - Changed the function name to prevent misunderstanding
> > > > > > >  - Added check for blk device
> > > > > > >  - Addressed the comments
> > > > > > > Changelog v3
> > > > > > >  - Split the function of the net device from vdpa_nl_cmd_dev_attr_set_doit
> > > > > > >  - Add a lock for the network device's dev_set_attr operation
> > > > > > >  - Address the comments
> > > > > > >
> > > > > > > Cindy Lu (2):
> > > > > > >   vdpa: support set mac address from vdpa tool
> > > > > > >   vdpa_sim_net: Add the support of set mac address
> > > > > > >
> > > > > > >  drivers/vdpa/vdpa.c                  | 81 ++++++++++++++++++++++++++++
> > > > > > >  drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 19 ++++++-
> > > > > > >  include/linux/vdpa.h                 |  9 ++++
> > > > > > >  include/uapi/linux/vdpa.h            |  1 +
> > > > > > >  4 files changed, 109 insertions(+), 1 deletion(-)
> > > > > > >
> > > > > > > --
> > > > > > > 2.45.0
> > > > > >
> > > > > > Mlx5 device already allows setting the mac and mtu during the vdpa device creation time.
> > > > > > Once the vdpa device is created, it binds to vdpa bus and other driver vhost_vdpa etc bind to it.
> > > > > > So there was no good reason in the past to support explicit config after device add complicate the flow for synchronizing this.
> > > > > >
> > > > > > The user who wants a device with new attributes, as well destroy and recreate the vdpa device with new desired attributes.
> > > > > >
> > > > > > vdpa_sim_net can also be extended for similar way when adding the vdpa device.
> > > > > >
> > > > > > Have you considered using the existing tool and kernel in place since 2021?
> > > > > > Such as commit d8ca2fa5be1.
> > > > > >
> > > > > > An example of it is,
> > > > > > $ vdpa dev add name bar mgmtdev vdpasim_net mac 00:11:22:33:44:55 mtu 9000
> > > > > >
> > > > > Hi Parav
> > > > > Really thanks for your comments. The reason for adding this function
> > > > > is to support Kubevirt.
> > > > > the problem we meet is that kubevirt chooses one random vdpa device
> > > > > from the pool and we don't know which one it going to pick. That means
> > > > > we can't get to know the Mac address before it is created. So we plan
> > > > > to have this function to change the mac address after it is created
> > > > > Thanks
> > > > > cindy
> > > >
> > > > Well you will need to change kubevirt to teach it to set
> > > > mac address, right?
> > >
> > > That's the plan. Adding Leonardo.
> > >
> > > Thanks
> >
> > So given you are going to change kubevirt, can we
> > change it to create devices as needed with the
> > existing API?
> >
> Hi Micheal and Parav,
> I'm really not familiar with kubevirt, hope Leonardo can help answer
> these questions
> Hi @Leonardo Milleri
> would you help answer these questions?
>
> Thanks
> Cindy
> > > >
> > > > --
> > > > MST
> > > >
> >
>

Powered by blists - more mailing lists