netdev - RE: [PATCH v10 00/20] dlb: introduce DLB device driver

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <BYAPR11MB309515F449B8660043A559E5D96C9@BYAPR11MB3095.namprd11.prod.outlook.com>
Date:   Mon, 15 Mar 2021 20:18:10 +0000
From:   "Chen, Mike Ximing" <mike.ximing.chen@...el.com>
To:     "Williams, Dan J" <dan.j.williams@...el.com>
CC:     Greg KH <gregkh@...uxfoundation.org>,
        Netdev <netdev@...r.kernel.org>,
        David Miller <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>,
        "Arnd Bergmann" <arnd@...db.de>,
        Pierre-Louis Bossart <pierre-louis.bossart@...ux.intel.com>,
        "Brandeburg, Jesse" <jesse.brandeburg@...el.com>
Subject: RE: [PATCH v10 00/20] dlb: introduce DLB device driver

> From: Dan Williams <dan.j.williams@...el.com>
> On Fri, Mar 12, 2021 at 1:55 PM Chen, Mike Ximing <mike.ximing.chen@...el.com> wrote:
> >
> > We support up to 16/32 VF/VDEVs (depending on version) with SRIOV and
> > SIOV. Role of the kernel driver includes VDEV Composition (vdcm
> > module), functional level reset, live migration, error handling, power
> > management, and etc..
> 
> Need some more specificity here. What about those features requires the kernel to get involved with a
> DLB2 specific ABI to manage ports, queues, credits, sequence numbers, etc...?

Role of the dlb kernel driver:

VDEV Composition
For example writing 1024 to the VDEV_CREDITS[0] register will allocate 1024 credits to VDEV 0. In this way, VFs or VDEVs can be composed  as mini-versions of the full device.
VDEV composition will leverage vfio-mdev to create the VDEV devices while the KMD will implement the VDCM.

Dynamic Composition
Such composition can be dynamic – the PF/VF interface supports scenarios whereby, for example, an application may wish to boost its credit allocation – can I have 100 more credits?

Functional Level Reset
Much of the internal storage is RAM based and not resettable by hardware schemes. There are also internal SRAM  based control structures (BCAM) that have to be flushed. 
The planned way to do this is, roughly:
  -- Kernel driver disables access from the associated ports  (to prevent any SW access, the application should be deadso this is a precaution).
  -- Kernel masquerades as the application to drain all data from internal queues. It can poll some internal counters to verify everything is fully drained.
  -- Only at this point can the resources associated with the VDEV be returned to the pool of available resources for handing to another application/VDEV.

Migration
Requirement is fairly similar to FLR. A VDEV has to be manually drained and reconstituted on another server, Kernel driver is responsible on both sides.

Error Handling
Errors include “Credit Excursions” where a VDEV attempts to use more of the internal capacity (credits) than has been allocated. In such a case, 
the data is dropped and an interrupt generated. All such interrupts are directed to the PF driver, which may simply forward them to a VF (via the PF/VF comms mechanism).

Power Management
The kernel driver keeps the device in D3Hot when not in use. The driver transitions the device to D0 when the first device file is opened or a VF or VDEV is created, 
and keeps it in that state until there are no open device files, memory mappings, or VFs/VDEVs.

Ioctl interface
Kernel driver provides ioctl interface for user applications to setup and configure dlb domains, ports, queues, scheduling types, credits, 
sequence numbers, and links between ports and queues.  Applications also use the interface to start, stop and inquire the dlb operations.

Thanks
Mike