netdev - Re: [net-next v5 13/15] devlink: Add devlink port documentation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20201215165758.4ff58f85@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Tue, 15 Dec 2020 16:57:58 -0800
From:   Jakub Kicinski <kuba@...nel.org>
To:     Saeed Mahameed <saeed@...nel.org>
Cc:     "David S. Miller" <davem@...emloft.net>,
        Jason Gunthorpe <jgg@...dia.com>,
        Leon Romanovsky <leonro@...dia.com>, netdev@...r.kernel.org,
        linux-rdma@...r.kernel.org, David Ahern <dsahern@...nel.org>,
        Jacob Keller <jacob.e.keller@...el.com>,
        Sridhar Samudrala <sridhar.samudrala@...el.com>,
        david.m.ertman@...el.com, dan.j.williams@...el.com,
        kiran.patil@...el.com, gregkh@...uxfoundation.org,
        Parav Pandit <parav@...dia.com>, Jiri Pirko <jiri@...dia.com>,
        Saeed Mahameed <saeedm@...dia.com>
Subject: Re: [net-next v5 13/15] devlink: Add devlink port documentation

On Tue, 15 Dec 2020 01:03:56 -0800 Saeed Mahameed wrote:
> +PCI controllers
> +---------------
> +In most cases a PCI device has only one controller. A controller consists of
> +potentially multiple physical and virtual functions. Such PCI function consists
> +of one or more ports.

s/Such//

you say consists in two consecutive sentences.

> This port of the function is represented by the devlink eswitch port.

"This port of the function"? Why not just "Each port"?

> +A PCI Device connected to multiple CPUs or multiple PCI root complexes or

Why is device capitalized all of the sudden?

> +SmartNIC, however, may have multiple controllers. For a device with multiple

a SmartNIC or SmartNICs

> +controllers, each controller is distinguished by a unique controller number.
> +An eswitch on the PCI device support ports of multiple controllers.

eswitch is on a PCI device?

> +An example view of a system with two controllers::
> +
> +                 ---------------------------------------------------------
> +                 |                                                       |
> +                 |           --------- ---------         ------- ------- |
> +    -----------  |           | vf(s) | | sf(s) |         |vf(s)| |sf(s)| |
> +    | server  |  | -------   ----/---- ---/----- ------- ---/--- ---/--- |
> +    | pci rc  |=== | pf0 |______/________/       | pf1 |___/_______/     |
> +    | connect |  | -------                       -------                 |
> +    -----------  |     | controller_num=1 (no eswitch)                   |
> +                 ------|--------------------------------------------------
> +                 (internal wire)
> +                       |
> +                 ---------------------------------------------------------
> +                 | devlink eswitch ports and reps                        |
> +                 | ----------------------------------------------------- |
> +                 | |ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 |ctrl-0 | |
> +                 | |pf0    | pf0vfN | pf0sfN | pf1    | pf1vfN |pf1sfN | |
> +                 | ----------------------------------------------------- |
> +                 | |ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 |ctrl-1 | |
> +                 | |pf0    | pf0vfN | pf0sfN | pf1    | pf1vfN |pf1sfN | |
> +                 | ----------------------------------------------------- |
> +                 |                                                       |
> +                 |                                                       |
> +    -----------  |           --------- ---------         ------- ------- |
> +    | smartNIC|  |           | vf(s) | | sf(s) |         |vf(s)| |sf(s)| |
> +    | pci rc  |==| -------   ----/---- ---/----- ------- ---/--- ---/--- |
> +    | connect |  | | pf0 |______/________/       | pf1 |___/_______/     |
> +    -----------  | -------                       -------                 |
> +                 |                                                       |
> +                 |  local controller_num=0 (eswitch)                     |
> +                 ---------------------------------------------------------
> +
> +In above example, external controller (identified by controller number = 1)
> +doesn't have eswitch. Local controller (identified by controller number = 0)
> +has the eswitch. Devlink instance on local controller has eswitch devlink
> +ports representing ports for both the controllers.
> +
> +Port function configuration
> +===========================
> +
> +A user can configure the port function attribute before enumerating the

s/A user/User/

/port function attribute/$something_meaningful/

> +PCI function. Usually it means, user should configure port function attribute

attributes, plural

> +before a bus specific device for the function is created. However, when
> +SRIOV is enabled, virtual function devices are created on the PCI bus.
> +Hence, function attribute should be configured before binding virtual
> +function device to the driver.
> +
> +User may set the hardware address of the function represented by the devlink
> +port function. For Ethernet port function this means a MAC address.