Date:   Wed, 16 Dec 2020 03:55:32 +0000
From:   Parav Pandit <parav@...dia.com>
To:     Jakub Kicinski <kuba@...nel.org>, Saeed Mahameed <saeed@...nel.org>
CC:     "David S. Miller" <davem@...emloft.net>,
        Jason Gunthorpe <jgg@...dia.com>,
        Leon Romanovsky <leonro@...dia.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        David Ahern <dsahern@...nel.org>,
        Jacob Keller <jacob.e.keller@...el.com>,
        Sridhar Samudrala <sridhar.samudrala@...el.com>,
        "david.m.ertman@...el.com" <david.m.ertman@...el.com>,
        "dan.j.williams@...el.com" <dan.j.williams@...el.com>,
        "kiran.patil@...el.com" <kiran.patil@...el.com>,
        "gregkh@...uxfoundation.org" <gregkh@...uxfoundation.org>,
        Saeed Mahameed <saeedm@...dia.com>
Subject: RE: [net-next v5 14/15] devlink: Extend devlink port documentation
 for subfunctions



> From: Jakub Kicinski <kuba@...nel.org>
> Sent: Wednesday, December 16, 2020 6:31 AM
> 
> On Tue, 15 Dec 2020 01:03:57 -0800 Saeed Mahameed wrote:
> > +Subfunctions are lightweight functions that has parent PCI function
> > +on which it is deployed. Subfunctions are created and deployed in
> > +unit of 1. Unlike SRIOV VFs, they don't require their own PCI virtual
> > +function. They communicate with the hardware through the parent PCI
> > +function. Subfunctions can possibly scale better.
> > +
> > +To use a subfunction, 3 steps setup sequence is followed.
> > +(1) create - create a subfunction;
> > +(2) configure - configure subfunction attributes;
> > +(3) deploy - deploy the subfunction;
> > +
> > +Subfunction management is done using devlink port user interface.
> > +User performs setup on the subfunction management device.
> > +
> > +(1) Create
> > +----------
> > +A subfunction is created using a devlink port interface. User adds
> > +the subfunction by adding a devlink port of subfunction flavour. The
> > +devlink kernel code calls down to subfunction management driver
> > +(devlink op) and asks it to create a subfunction devlink port. Driver
> > +then instantiates the subfunction port and any associated objects
> > +such as health reporters and representor netdevice.
> > +
> > +(2) Configure
> > +-------------
> > +Subfunction devlink port is created but it is not active yet. That
> > +means the entities are created on devlink side, the e-switch port
> > +representor is created, but the subfunction device itself it not
> > +created. User might use e-switch port representor to do settings,
> > +putting it into bridge, adding TC rules, etc. User might as well
> > +configure the hardware address (such as MAC address) of the subfunction
> while subfunction is inactive.
> > +
> > +(3) Deploy
> > +----------
> > +Once subfunction is configured, user must activate it to use it. Upon
> > +activation, subfunction management driver asks the subfunction
> > +management device to instantiate the actual subfunction device on
> particular PCI function.
> > +A subfunction device is created on the
> > +:ref:`Documentation/driver-api/auxiliary_bus.rst <auxiliary_bus>`. At this
> point matching subfunction driver binds to the subfunction's auxiliary device.
> > +
> > +Terms and Definitions
> > +=====================
> > +
> > +.. list-table:: Terms and Definitions
> > +   :widths: 22 90
> > +
> > +   * - Term
> > +     - Definitions
> > +   * - ``PCI device``
> > +     - A physical PCI device having one or more PCI bus consists of one or
> > +       more PCI controllers.
> > +   * - ``PCI controller``
> > +     -  A controller consists of potentially multiple physical functions,
> > +        virtual functions and subfunctions.
> > +   * - ``Port function``
> > +     -  An object to manage the function of a port.
> > +   * - ``Subfunction``
> > +     -  A lightweight function that has parent PCI function on which it is
> > +        deployed.
> > +   * - ``Subfunction device``
> > +     -  A bus device of the subfunction, usually on a auxiliary bus.
> > +   * - ``Subfunction driver``
> > +     -  A device driver for the subfunction auxiliary device.
> > +   * - ``Subfunction management device``
> > +     -  A PCI physical function that supports subfunction management.
> > +   * - ``Subfunction management driver``
> > +     -  A device driver for PCI physical function that supports
> > +        subfunction management using devlink port interface.
> > +   * - ``Subfunction host driver``
> > +     -  A device driver for PCI physical function that host subfunction
> > +        devices. In most cases it is same as subfunction management driver.
> When
> > +        subfunction is used on external controller, subfunction management
> and
> > +        host drivers are different.
> 
> Would be great if someone from Mellanox could proof read this before we
> spend cycles on correcting spelling in public review.
Will get it done.

Powered by blists - more mailing lists