Date:   Tue, 19 Nov 2019 14:51:29 +0800
From:   Jason Wang <jasowang@...hat.com>
To:     Parav Pandit <parav@...lanox.com>,
        Jeff Kirsher <jeffrey.t.kirsher@...el.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "gregkh@...uxfoundation.org" <gregkh@...uxfoundation.org>
Cc:     Dave Ertman <david.m.ertman@...el.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        "nhorman@...hat.com" <nhorman@...hat.com>,
        "sassmann@...hat.com" <sassmann@...hat.com>,
        "jgg@...pe.ca" <jgg@...pe.ca>, Kiran Patil <kiran.patil@...el.com>,
        "Michael S. Tsirkin" <mst@...hat.com>,
        Alex Williamson <alex.williamson@...hat.com>,
        "Bie, Tiwei" <tiwei.bie@...el.com>
Subject: Re: [net-next v2 1/1] virtual-bus: Implementation of Virtual Bus


On 2019/11/19 下午12:36, Parav Pandit wrote:
> Hi Jason Wang,
>
>> From: Jason Wang <jasowang@...hat.com>
>> Sent: Monday, November 18, 2019 10:08 PM
>>
>> On 2019/11/16 上午7:25, Parav Pandit wrote:
>>> Hi Jeff,
>>>
>>>> From: Jeff Kirsher <jeffrey.t.kirsher@...el.com>
>>>> Sent: Friday, November 15, 2019 4:34 PM
>>>>
>>>> From: Dave Ertman <david.m.ertman@...el.com>
>>>>
>>>> This is the initial implementation of the Virtual Bus, virtbus_device
>>>> and virtbus_driver.  The virtual bus is a software based bus intended
>>>> to support lightweight devices and drivers and provide matching
>>>> between them and probing of the registered drivers.
>>>>
>>>> The primary purpose of the virual bus is to provide matching services
>>>> and to pass the data pointer contained in the virtbus_device to the
>>>> virtbus_driver during its probe call.  This will allow two separate
>>>> kernel objects to match up and start communication.
>>>>
>>> It is fundamental to know that rdma device created by virtbus_driver will be
>> anchored to which bus for an non abusive use.
>>> virtbus or parent pci bus?
>>> I asked this question in v1 version of this patch.
>>>
>>> Also since it says - 'to support lightweight devices', documenting that
>> information is critical to avoid ambiguity.
>>> Since for a while I am working on the subbus/subdev_bus/xbus/mdev [1]
>> whatever we want to call it, it overlaps with your comment about 'to support
>> lightweight devices'.
>>> Hence let's make things crystal clear weather the purpose is 'only matching
>> service' or also 'lightweight devices'.
>>> If this is only matching service, lets please remove lightweight devices part..
>>
>> Yes, if it's matching + lightweight device, its function is almost a duplication of
>> mdev. And I'm working on extending mdev[1] to be a generic module to
>> support any types of virtual devices a while. The advantage of mdev is:
>>
>> 1) ready for the userspace driver (VFIO based)
>> 2) have a sysfs/GUID based management interface
>>
>> So for 1, it's not clear that how userspace driver would be supported here, or
>> it's completely not being accounted in this series? For 2, it looks to me that this
>> series leave it to the implementation, this means management to learn several
>> vendor specific interfaces which seems a burden.
>>
>> Note, technically Virtual Bus could be implemented on top of [1] with the full
>> lifecycle API.
>>
>> [1] https://lkml.org/lkml/2019/11/18/261
>>
>>
>>> You additionally need modpost support for id table integration to modifo,
>> modprobe and other tools.
>>> A small patch similar to this one [2] is needed.
>>> Please include in the series.
>>>
>>> [..]
>>
>> And probably a uevent method. But rethinking of this, matching through a
>> single virtual bus seems not good. What if driver want to do some specific
>> matching? E.g for virtio, we may want a vhost-net driver that only match
>> networking device. With a single bus, it probably means you need another bus
>> on top and provide the virtio specific matching there.
>> This looks not straightforward as allowing multiple type of buses.
>>
> The purpose of the bus is to attach two drivers,


Right, I just start to think whether it was generic to support the case 
as virtio or mdev to avoid function duplications.


>   mlx5_core (creator of netdevices) and mlx5_ib (create of rdma devices) on single PCI function.
> Meaning 'multiple classes of devices' are created on top of single underlying parent device.


This is not what I read, the doc said:

"
+One use case example is an rdma driver needing to connect with several
+different types of PCI LAN devices to be able to request resources from
+them (queue sets).  Each LAN driver that supports rdma will register a
+virtbus_device on the virtual bus for each physical function. The rdma
+driver will register as a virtbus_driver on the virtual bus to be
+matched up with multiple virtbus_devices and receive a pointer to a
+struct containing the callbacks that the PCI LAN drivers support for
+registering with them.

"

So it means to connect a single rdma driver with several RDMA capable 
LAN drivers on top of several PCI functions. If this is true, I'm not 
quite sure the advantage of using a bus since it's more like aggregation 
as what bond/team did.


>
> So bus is just the 'matching service' and nothing more. It is not meant to address virtio, mdev, sub functions usecases.


Probably, for virtio mdev we need more than just matching: life cycle 
management, cooperation with VFIO and we also want to be prepared for 
the device slicing (like sub functions).

Thanks

Powered by blists - more mailing lists