| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Wed, 16 Mar 2022 03:08:14 -0400
From: Tianfei Zhang <tianfei.zhang@...el.com>
To: hao.wu@...el.com, trix@...hat.com, mdf@...nel.org,
yilun.xu@...el.com, linux-fpga@...r.kernel.org,
linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
rdunlap@...radead.org
Cc: corbet@....net, Tianfei zhang <tianfei.zhang@...el.com>
Subject: [PATCH v6 6/6] Documentation: fpga: dfl: add description of OFS
From: Tianfei zhang <tianfei.zhang@...el.com>
This patch adds description about OFS support for DFL.
---
v6:
fix documentation with Randy's comment.
v5:
fix documentation with Matthew and Randy's comment.
v4:
add description about access the AFU on "multiple VFs per PR slot" model.
v3:
change IOFS to OFS in documentation.
v2:
* Fixs some typos.
* Adds more detail description about the models of AFU access which supported in OFS.
Signed-off-by: Tianfei zhang <tianfei.zhang@...el.com>
---
Documentation/fpga/dfl.rst | 114 +++++++++++++++++++++++++++++++++++++
1 file changed, 114 insertions(+)
diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst
index ef9eec71f6f3..93f262fe7b8c 100644
--- a/Documentation/fpga/dfl.rst
+++ b/Documentation/fpga/dfl.rst
@@ -556,6 +556,120 @@ new DFL feature via UIO direct access, its feature id should be added to the
driver's id_table.
+Open FPGA Stack
+=====================
+
+Open FPGA Stack (OFS) is a collection of RTL and open source software providing
+interfaces to access the instantiated RTL easily in an FPGA. OFS leverages the
+DFL for the implementation of the FPGA RTL design.
+
+OFS designs allow for the arrangement of software interfaces across multiple
+PCIe endpoints. Some of these interfaces may be PFs defined in the static region
+that connect to interfaces in an IP that is loaded via Partial Reconfiguration (PR).
+And some of these interfaces may be VFs defined in the PR region that can be
+reconfigured by the end-user. Furthermore, these PFs/VFs may use DFLs such that
+features may be discovered and accessed in user space (with the aid of a generic
+kernel driver like vfio-pci). The diagram below depicts an example design with two
+PFs and two VFs. In this example, it will export the management functions via PF0,
+PF1 will bind with virtio-net driver presenting itself as a network interface to
+the OS. The other functions, VF0 and VF1, leverage VFIO to export the MMIO space
+to an application or assign to a VM.
+::
+
+ +-----------------+ +--------------+ +-------------+ +------------+
+ | FPGA Management | | VirtIO | | User App | | Virtual |
+ | App | | App | | | | Machine |
+ +--------+--------+ +------+-------+ +------+------+ +-----+------+
+ | | | |
+ +--------+--------+ +------+-------+ +------+------+ |
+ | DFL Driver | |VirtIO driver | | VFIO | |
+ +--------+--------+ +------+-------+ +------+------+ |
+ | | | |
+ | | | |
+ +--------+--------+ +------+-------+ +------+------+ +----+------+
+ | PF0 | | PF1 | | PF0_VF0 | | PF0_VF1 |
+ +-----------------+ +--------------+ +-------------+ +-----------+
+
+As accelerators are specialized hardware, they are typically limited in the
+number installed in a given system. Many use cases require them to be shared
+across multiple software contexts or threads of software execution, either
+through partitioning of individual dedicated resources, or virtualization of
+shared resources. OFS provides several models to share the AFU resources via
+PR mechanism and hardware-based virtualization schemes.
+
+1. Legacy model.
+ With legacy model FPGA cards like Intel PAC N3000 or N5000, there is
+ a notion that the boundary between the AFU and the shell is also the unit of
+ PR for those FPGA platforms. This model is only able to handle a
+ single context, because it only has one PR engine, and one PR region which
+ has an associated Port device.
+2. Multiple VFs per PR slot.
+ In this model, available AFU resources may allow instantiation of many VFs
+ which have a dedicated PCIe function with their own dedicated MMIO space, or
+ partition a region of MMIO space on a single PCIe function. Intel PAC N6000
+ card has implemented this model.
+ In this model, the AFU/PR slot was not connected to port device. For DFL's view,
+ the Next_AFU pointer in FIU feature header of port device points to NULL in this
+ model, so in AFU driver perspective, there is no AFU MMIO region managed by
+ AFU driver. On the other hand, each VF can start with an AFU feature header without
+ being connected to a FIU Port feature header.
+
+In multiple VFs per PR slot model, the port device can still be accessed using
+ioctls API which expose /dev/dfl-port.h device nodes, like port reset, get
+port info, whose APIs were mentioned in AFU section in this documentation. But
+it cannot access the AFU MMIO space via AFU ioctl APIs like DFL_FPGA_PORT_DMA_MAP
+because there is no AFU MMIO space managed in the AFU driver. Users can access
+the AFU resource by creating VF devices via PCIe SRIOV interface, and then access
+the VF via VFIO driver or assign the VF to VM.
+
+In multiple VFs per PR slot model, the steps to enable VFs are compatible with
+legacy mode which are mentioned in "FPGA virtualization - PCIe SRIOV" section
+in this documentation.
+
+OFS provides the diversity for accessing the AFU resource to RTL developer.
+An IP designer may choose to add more than one PF for interfacing with IP
+on the FPGA and choose different model to access the AFU resource.
+
+There is one reference architecture design using the "Multiple VFs per PR slot"
+model for OFS as illustrated below. In this reference design, it exports the
+FPGA management functions via PF0. PF1 will bind with virtio-net driver
+presenting itself as a network interface to the OS. PF2 will bind to the
+vfio-pci driver allowing the user space software to discover and interface
+with the specific workload like diagnostic test. To access the AFU resource,
+it uses SR-IOV to partition workload interfaces across various VFs.
+::
+
+ +----------------------+
+ | PF/VF mux/demux |
+ +--+--+-----+------+-+-+
+ | | | | |
+ +------------------------+ | | | |
+ PF0 | +---------+ +-+ | |
+ +---+---+ | +---+----+ | |
+ | DFH | | | DFH | | |
+ +-------+ +-----+----+ +--------+ | |
+ | FME | | VirtIO | | Test | | |
+ +---+---+ +----------+ +--------+ | |
+ | PF1 PF2 | |
+ | | |
+ | +----------+ |
+ | | ++
+ | | |
+ | | PF0_VF0 | PF0_VF1
+ | +-----------------+-----------+------------+
+ | | +-----+-----------+--------+ |
+ | | | | | | |
+ | | +------+ | +--+ -+ +--+---+ | |
+ | | | Port | | | DFH | | DFH | | |
+ +-----------+ +------+ | +-----+ +------+ | |
+ | | | DEV | | DEV | | |
+ | | +-----+ +------+ | |
+ | | PR Slot | |
+ | +--------------------------+ |
+ | Port Gasket |
+ +------------------------------------------+
+
+
Open discussion
===============
FME driver exports one ioctl (DFL_FPGA_FME_PORT_PR) for partial reconfiguration
--
2.26.2
Powered by blists - more mailing lists