Date:   Fri, 24 Mar 2023 17:27:06 -0700
From:   Shannon Nelson <shannon.nelson@....com>
To:     Jason Wang <jasowang@...hat.com>
Cc:     mst@...hat.com, virtualization@...ts.linux-foundation.org,
        brett.creeley@....com, davem@...emloft.net, netdev@...r.kernel.org,
        kuba@...nel.org, drivers@...sando.io
Subject: Re: [PATCH v3 virtio 8/8] pds_vdpa: pds_vdps.rst and Kconfig

On 3/22/23 10:22 PM, Jason Wang wrote:
> On Thu, Mar 23, 2023 at 3:11 AM Shannon Nelson <shannon.nelson@....com> wrote:
>>
>> Add the documentation and Kconfig entry for pds_vdpa driver.
>>
>> Signed-off-by: Shannon Nelson <shannon.nelson@....com>
>> ---
>>   .../device_drivers/ethernet/amd/pds_vdpa.rst  | 84 +++++++++++++++++++
>>   .../device_drivers/ethernet/index.rst         |  1 +
> 
> I wonder if it's better to have a dedicated directory for vDPA.

It probably makes sense, but I wasn't going to be so bold as to start 
messing with the documentation layout.

> 
>>   MAINTAINERS                                   |  4 +
>>   drivers/vdpa/Kconfig                          |  8 ++
>>   4 files changed, 97 insertions(+)
>>   create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst
>>
>> diff --git a/Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst b/Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst
>> new file mode 100644
>> index 000000000000..d41f6dd66e3e
>> --- /dev/null
>> +++ b/Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst
>> @@ -0,0 +1,84 @@
>> +.. SPDX-License-Identifier: GPL-2.0+
>> +.. note: can be edited and viewed with /usr/bin/formiko-vim
>> +
>> +==========================================================
>> +PCI vDPA driver for the AMD/Pensando(R) DSC adapter family
>> +==========================================================
>> +
>> +AMD/Pensando vDPA VF Device Driver
>> +Copyright(c) 2023 Advanced Micro Devices, Inc
>> +
>> +Overview
>> +========
>> +
>> +The ``pds_vdpa`` driver is an auxiliary bus driver that supplies
>> +a vDPA device for use by the virtio network stack.  It is used with
>> +the Pensando Virtual Function devices that offer vDPA and virtio queue
>> +services.  It depends on the ``pds_core`` driver and hardware for the PF
>> +and VF PCI handling as well as for device configuration services.
>> +
>> +Using the device
>> +================
>> +
>> +The ``pds_vdpa`` device is enabled via multiple configuration steps and
>> +depends on the ``pds_core`` driver to create and enable SR-IOV Virtual
>> +Function devices.
>> +
>> +Shown below are the steps to bind the driver to a VF and also to the
>> +associated auxiliary device created by the ``pds_core`` driver.
>> +
>> +.. code-block:: bash
>> +
>> +  #!/bin/bash
>> +
>> +  modprobe pds_core
>> +  modprobe vdpa
>> +  modprobe pds_vdpa
>> +
>> +  PF_BDF=`grep -H "vDPA.*1" /sys/kernel/debug/pds_core/*/viftypes | head -1 | awk -F / '{print $6}'`
> 
> This seems to require debugfs, I wonder if it's better to switch to
> using /sys/bus/pci ?

This was a quick and dirty way to find any PF devices that supported our 
vDPA.  Yes, this should get replaced with a non-debugfs method, I'll 
come up with something nicer.

Thanks,
sln

> 
> Others look good.
> 
> Thanks
> 
>> +
>> +  # Enable vDPA VF auxiliary device(s) in the PF
>> +  devlink dev param set pci/$PF_BDF name enable_vnet value true cmode runtime
>> +
>> +  # Create a VF for vDPA use
>> +  echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs
>> +
>> +  # Find the vDPA services/devices available
>> +  PDS_VDPA_MGMT=`vdpa mgmtdev show | grep vDPA | head -1 | cut -d: -f1`
>> +
>> +  # Create a vDPA device for use in virtio network configurations
>> +  vdpa dev add name vdpa1 mgmtdev $PDS_VDPA_MGMT mac 00:11:22:33:44:55
>> +
>> +  # Set up an ethernet interface on the vdpa device
>> +  modprobe virtio_vdpa
>> +
>> +
>> +
>> +Enabling the driver
>> +===================
>> +
>> +The driver is enabled via the standard kernel configuration system,
>> +using the make command::
>> +
>> +  make oldconfig/menuconfig/etc.
>> +
>> +The driver is located in the menu structure at:
>> +
>> +  -> Device Drivers
>> +    -> Network device support (NETDEVICES [=y])
>> +      -> Ethernet driver support
>> +        -> Pensando devices
>> +          -> Pensando Ethernet PDS_VDPA Support
>> +
>> +Support
>> +=======
>> +
>> +For general Linux networking support, please use the netdev mailing
>> +list, which is monitored by Pensando personnel::
>> +
>> +  netdev@...r.kernel.org
>> +
>> +For more specific support needs, please use the Pensando driver support
>> +email::
>> +
>> +  drivers@...sando.io
>> diff --git a/Documentation/networking/device_drivers/ethernet/index.rst b/Documentation/networking/device_drivers/ethernet/index.rst
>> index eaaf284e69e6..88dd38c7eb6d 100644
>> --- a/Documentation/networking/device_drivers/ethernet/index.rst
>> +++ b/Documentation/networking/device_drivers/ethernet/index.rst
>> @@ -14,6 +14,7 @@ Contents:
>>      3com/vortex
>>      amazon/ena
>>      amd/pds_core
>> +   amd/pds_vdpa
>>      altera/altera_tse
>>      aquantia/atlantic
>>      chelsio/cxgb
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 95b5f25a2c06..2af133861068 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -22108,6 +22108,10 @@ SNET DPU VIRTIO DATA PATH ACCELERATOR
>>   R:     Alvaro Karsz <alvaro.karsz@...id-run.com>
>>   F:     drivers/vdpa/solidrun/
>>
>> +PDS DSC VIRTIO DATA PATH ACCELERATOR
>> +R:     Shannon Nelson <shannon.nelson@....com>
>> +F:     drivers/vdpa/pds/
>> +
>>   VIRTIO BALLOON
>>   M:     "Michael S. Tsirkin" <mst@...hat.com>
>>   M:     David Hildenbrand <david@...hat.com>
>> diff --git a/drivers/vdpa/Kconfig b/drivers/vdpa/Kconfig
>> index cd6ad92f3f05..c910cb119c1b 100644
>> --- a/drivers/vdpa/Kconfig
>> +++ b/drivers/vdpa/Kconfig
>> @@ -116,4 +116,12 @@ config ALIBABA_ENI_VDPA
>>            This driver includes a HW monitor device that
>>            reads health values from the DPU.
>>
>> +config PDS_VDPA
>> +       tristate "vDPA driver for AMD/Pensando DSC devices"
>> +       depends on PDS_CORE
>> +       help
>> +         VDPA network driver for AMD/Pensando's PDS Core devices.
>> +         With this driver, the VirtIO dataplane can be
>> +         offloaded to an AMD/Pensando DSC device.
>> +
>>   endif # VDPA
>> --
>> 2.17.1
>>
> 

Powered by blists - more mailing lists