Message-ID: <1aa2c865-3a9b-441a-ba61-b551f3f7a832@amd.com>
Date: Thu, 18 Jan 2024 13:34:16 +0100
From: Michal Simek <michal.simek@....com>
To: Rob Herring <robh@...nel.org>
Cc: linux-kernel@...r.kernel.org, monstr@...str.eu, michal.simek@...inx.com,
 git@...inx.com, Conor Dooley <conor+dt@...nel.org>,
 Krzysztof Kozlowski <krzysztof.kozlowski+dt@...aro.org>,
 Moritz Fischer <mdf@...nel.org>, Tom Rix <trix@...hat.com>,
 Wu Hao <hao.wu@...el.com>, Xu Yilun <yilun.xu@...el.com>,
 "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS"
 <devicetree@...r.kernel.org>,
 "open list:FPGA MANAGER FRAMEWORK" <linux-fpga@...r.kernel.org>
Subject: Re: [PATCH v3] dt-bindings: fpga: Convert fpga-region binding to yaml



On 1/17/24 22:47, Rob Herring wrote:
> On Wed, Jan 17, 2024 at 12:30:58PM +0100, Michal Simek wrote:
>> Convert the generic fpga region DT binding to json-schema.
>> There are some differences compare to txt version.
>> 1. DT overlay can't be described in example that's why directly include
>> information from overlay to node which was referenced. It is visible in
>> example with /* DT Overlay contains: &... */
>>
>> 2. All example have been rewritten to be simpler and describe only full
>> reconfiguration and partial reconfiguration with one bridge.
>> Completely drop the case where fpga region can inside partial
>> reconfiguration region which is already described in description
>>
>> 3. Fixed some typos in descriptions compare to txt version but most of it
>> is just c&p from txt file.
>>
>> Signed-off-by: Michal Simek <michal.simek@....com>
>> ---
>>
>> Changes in v3:
>> - drop fpga bridge and mgr descriptions in example
>> - use additionalProperties: true
>> - use fixed-factor-clock instead
>> - fixed matching pattern
>>
>> Changes in v2:
>> - Fix typo in subject
>> - Fix comment in bridge example
>> - Change license back to gpl-2.0 only
>> - Do not define firware-name type and add maxItems 1
>> - Make fpga-bridge phandle-array
>> - Drop ranges property because of missing reg property of fpga-region
>> - Also describe case with fixed clock node and axi bus
>> - Fix fpga-region names in example
>>
>> Please let me know if there is a way to describe overlays to dt root to be
>> able to reference fpga region back.
> 
> No. Overlays are somewhat outside the scope of schema. Certainly not
> supported in the examples. When the dts files are free of warnings I'll
> worry about overlays. :)
> 
>> ---
>>   .../devicetree/bindings/fpga/fpga-region.txt  | 479 ------------------
>>   .../devicetree/bindings/fpga/fpga-region.yaml | 351 +++++++++++++
>>   2 files changed, 351 insertions(+), 479 deletions(-)
>>   delete mode 100644 Documentation/devicetree/bindings/fpga/fpga-region.txt
>>   create mode 100644 Documentation/devicetree/bindings/fpga/fpga-region.yaml
> 
> 
>> diff --git a/Documentation/devicetree/bindings/fpga/fpga-region.yaml b/Documentation/devicetree/bindings/fpga/fpga-region.yaml
>> new file mode 100644
>> index 000000000000..fb6fe6f43fa8
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/fpga/fpga-region.yaml
>> @@ -0,0 +1,351 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/fpga/fpga-region.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: FPGA Region
>> +
>> +maintainers:
>> +  - Michal Simek <michal.simek@....com>
>> +
>> +description: |
>> +  CONTENTS
>> +   - Introduction
>> +   - Terminology
>> +   - Sequence
>> +   - FPGA Region
>> +   - Supported Use Models
>> +   - Constraints
>> +
>> +
>> +  Introduction
>> +  ============
>> +
>> +  FPGA Regions represent FPGA's and partial reconfiguration regions of FPGA's in
>> +  the Device Tree.  FPGA Regions provide a way to program FPGAs under device tree
>> +  control.
>> +
>> +  The documentation hits some of the high points of FPGA usage and
>> +  attempts to include terminology used by both major FPGA manufacturers.  This
>> +  document isn't a replacement for any manufacturers specifications for FPGA
>> +  usage.
>> +
>> +
>> +  Terminology
>> +  ===========
>> +
>> +  Full Reconfiguration
>> +   * The entire FPGA is programmed.
>> +
>> +  Partial Reconfiguration (PR)
>> +   * A section of an FPGA is reprogrammed while the rest of the FPGA is not
>> +     affected.
>> +   * Not all FPGA's support PR.
>> +
>> +  Partial Reconfiguration Region (PRR)
>> +   * Also called a "reconfigurable partition"
>> +   * A PRR is a specific section of an FPGA reserved for reconfiguration.
>> +   * A base (or static) FPGA image may create a set of PRR's that later may
>> +     be independently reprogrammed many times.
>> +   * The size and specific location of each PRR is fixed.
>> +   * The connections at the edge of each PRR are fixed.  The image that is loaded
>> +     into a PRR must fit and must use a subset of the region's connections.
>> +   * The busses within the FPGA are split such that each region gets its own
>> +     branch that may be gated independently.
>> +
>> +  Persona
>> +   * Also called a "partial bit stream"
>> +   * An FPGA image that is designed to be loaded into a PRR.  There may be
>> +     any number of personas designed to fit into a PRR, but only one at a time
>> +     may be loaded.
>> +   * A persona may create more regions.
>> +
>> +  FPGA Bridge
>> +   * FPGA Bridges gate bus signals between a host and FPGA.
>> +   * FPGA Bridges should be disabled while the FPGA is being programmed to
>> +     prevent spurious signals on the cpu bus and to the soft logic.
>> +   * FPGA bridges may be actual hardware or soft logic on an FPGA.
>> +   * During Full Reconfiguration, hardware bridges between the host and FPGA
>> +     will be disabled.
>> +   * During Partial Reconfiguration of a specific region, that region's bridge
>> +     will be used to gate the busses.  Traffic to other regions is not affected.
>> +   * In some implementations, the FPGA Manager transparently handles gating the
>> +     buses, eliminating the need to show the hardware FPGA bridges in the
>> +     device tree.
>> +   * An FPGA image may create a set of reprogrammable regions, each having its
>> +     own bridge and its own split of the busses in the FPGA.
>> +
>> +  FPGA Manager
>> +   * An FPGA Manager is a hardware block that programs an FPGA under the control
>> +     of a host processor.
>> +
>> +  Base Image
>> +   * Also called the "static image"
>> +   * An FPGA image that is designed to do full reconfiguration of the FPGA.
>> +   * A base image may set up a set of partial reconfiguration regions that may
>> +     later be reprogrammed.
>> +
>> +      ----------------       ----------------------------------
>> +      |  Host CPU    |       |             FPGA               |
>> +      |              |       |                                |
>> +      |          ----|       |       -----------    --------  |
>> +      |          | H |       |   |==>| Bridge0 |<==>| PRR0 |  |
>> +      |          | W |       |   |   -----------    --------  |
>> +      |          |   |       |   |                            |
>> +      |          | B |<=====>|<==|   -----------    --------  |
>> +      |          | R |       |   |==>| Bridge1 |<==>| PRR1 |  |
>> +      |          | I |       |   |   -----------    --------  |
>> +      |          | D |       |   |                            |
>> +      |          | G |       |   |   -----------    --------  |
>> +      |          | E |       |   |==>| Bridge2 |<==>| PRR2 |  |
>> +      |          ----|       |       -----------    --------  |
>> +      |              |       |                                |
>> +      ----------------       ----------------------------------
>> +
>> +  Figure 1: An FPGA set up with a base image that created three regions.  Each
>> +  region (PRR0-2) gets its own split of the busses that is independently gated by
>> +  a soft logic bridge (Bridge0-2) in the FPGA.  The contents of each PRR can be
>> +  reprogrammed independently while the rest of the system continues to function.
>> +
>> +
>> +  Sequence
>> +  ========
>> +
>> +  When a DT overlay that targets an FPGA Region is applied, the FPGA Region will
>> +  do the following:
>> +
>> +   1. Disable appropriate FPGA bridges.
>> +   2. Program the FPGA using the FPGA manager.
>> +   3. Enable the FPGA bridges.
>> +   4. The Device Tree overlay is accepted into the live tree.
>> +   5. Child devices are populated.
>> +
>> +  When the overlay is removed, the child nodes will be removed and the FPGA Region
>> +  will disable the bridges.
>> +
>> +
>> +  FPGA Region
>> +  ===========
>> +
>> +  FPGA Regions represent FPGA's and FPGA PR regions in the device tree.  An FPGA
>> +  Region brings together the elements needed to program on a running system and
>> +  add the child devices:
>> +
>> +   * FPGA Manager
>> +   * FPGA Bridges
>> +   * image-specific information needed to the programming.
>> +   * child nodes
>> +
>> +  The intended use is that a Device Tree overlay (DTO) can be used to reprogram an
>> +  FPGA while an operating system is running.
>> +
>> +  An FPGA Region that exists in the live Device Tree reflects the current state.
>> +  If the live tree shows a "firmware-name" property or child nodes under an FPGA
>> +  Region, the FPGA already has been programmed.  A DTO that targets an FPGA Region
>> +  and adds the "firmware-name" property is taken as a request to reprogram the
>> +  FPGA.  After reprogramming is successful, the overlay is accepted into the live
>> +  tree.
>> +
>> +  The base FPGA Region in the device tree represents the FPGA and supports full
>> +  reconfiguration.  It must include a phandle to an FPGA Manager.  The base
>> +  FPGA region will be the child of one of the hardware bridges (the bridge that
>> +  allows register access) between the cpu and the FPGA.  If there are more than
>> +  one bridge to control during FPGA programming, the region will also contain a
>> +  list of phandles to the additional hardware FPGA Bridges.
>> +
>> +  For partial reconfiguration (PR), each PR region will have an FPGA Region.
>> +  These FPGA regions are children of FPGA bridges which are then children of the
>> +  base FPGA region.  The "Full Reconfiguration to add PRR's" example below shows
>> +  this.
>> +
>> +  If an FPGA Region does not specify an FPGA Manager, it will inherit the FPGA
>> +  Manager specified by its ancestor FPGA Region.  This supports both the case
>> +  where the same FPGA Manager is used for all of an FPGA as well the case where
>> +  a different FPGA Manager is used for each region.
>> +
>> +  FPGA Regions do not inherit their ancestor FPGA regions' bridges.  This prevents
>> +  shutting down bridges that are upstream from the other active regions while one
>> +  region is getting reconfigured (see Figure 1 above).  During PR, the FPGA's
>> +  hardware bridges remain enabled.  The PR regions' bridges will be FPGA bridges
>> +  within the static image of the FPGA.
>> +
>> +
>> +  Supported Use Models
>> +  ====================
>> +
>> +  In all cases the live DT must have the FPGA Manager, FPGA Bridges (if any), and
>> +  a FPGA Region.  The target of the Device Tree Overlay is the FPGA Region.  Some
>> +  uses are specific to an FPGA device.
>> +
>> +   * No FPGA Bridges
>> +     In this case, the FPGA Manager which programs the FPGA also handles the
>> +     bridges behind the scenes.  No FPGA Bridge devices are needed for full
>> +     reconfiguration.
>> +
>> +   * Full reconfiguration with hardware bridges
>> +     In this case, there are hardware bridges between the processor and FPGA that
>> +     need to be controlled during full reconfiguration.  Before the overlay is
>> +     applied, the live DT must include the FPGA Manager, FPGA Bridges, and a
>> +     FPGA Region.  The FPGA Region is the child of the bridge that allows
>> +     register access to the FPGA.  Additional bridges may be listed in a
>> +     fpga-bridges property in the FPGA region or in the device tree overlay.
>> +
>> +   * Partial reconfiguration with bridges in the FPGA
>> +     In this case, the FPGA will have one or more PRR's that may be programmed
>> +     separately while the rest of the FPGA can remain active.  To manage this,
>> +     bridges need to exist in the FPGA that can gate the buses going to each FPGA
>> +     region while the buses are enabled for other sections.  Before any partial
>> +     reconfiguration can be done, a base FPGA image must be loaded which includes
>> +     PRR's with FPGA bridges.  The device tree should have an FPGA region for each
>> +     PRR.
>> +
>> +  Constraints
>> +  ===========
>> +
>> +  It is beyond the scope of this document to fully describe all the FPGA design
>> +  constraints required to make partial reconfiguration work[1] [2] [3], but a few
>> +  deserve quick mention.
>> +
>> +  A persona must have boundary connections that line up with those of the partition
>> +  or region it is designed to go into.
>> +
>> +  During programming, transactions through those connections must be stopped and
>> +  the connections must be held at a fixed logic level.  This can be achieved by
>> +  FPGA Bridges that exist on the FPGA fabric prior to the partial reconfiguration.
>> +
>> +  --
>> +  [1] www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/ug/ug_partrecon.pdf
>> +  [2] tspace.library.utoronto.ca/bitstream/1807/67932/1/Byma_Stuart_A_201411_MAS_thesis.pdf
>> +  [3] https://www.xilinx.com/support/documentation/sw_manuals/xilinx14_1/ug702.pdf
>> +
>> +properties:
>> +  $nodename:
>> +    pattern: "^fpga-region(-([0-9]|[1-9][0-9]+))?$"
>> +
>> +  compatible:
>> +    const: fpga-region
>> +
>> +  "#address-cells": true
>> +  "#size-cells": true
>> +
>> +  config-complete-timeout-us:
>> +    description:
>> +      The maximum time in microseconds time for the FPGA to go to operating
>> +      mode after the region has been programmed.
>> +
>> +  encrypted-fpga-config:
>> +    type: boolean
>> +    description:
>> +      Set if the bitstream is encrypted.
>> +
>> +  external-fpga-config:
>> +    type: boolean
>> +    description:
>> +      Set if the FPGA has already been configured prior to OS boot up.
>> +
>> +  firmware-name:
>> +    maxItems: 1
>> +    description:
>> +      Should contain the name of an FPGA image file located on the firmware
>> +      search path. If this property shows up in a live device tree it indicates
>> +      that the FPGA has already been programmed with this image.
>> +      If this property is in an overlay targeting an FPGA region, it is
>> +      a request to program the FPGA with that image.
>> +
>> +  fpga-bridges:
>> +    $ref: /schemas/types.yaml#/definitions/phandle-array
>> +    description:
>> +      Should contain a list of phandles to FPGA Bridges that must be
>> +      controlled during FPGA programming along with the parent FPGA bridge.
>> +      This property is optional if the FPGA Manager handles the bridges.
>> +      If the fpga-region is  the child of an fpga-bridge, the list should not
>> +      contain the parent bridge.
>> +
>> +  fpga-mgr:
>> +    $ref: /schemas/types.yaml#/definitions/phandle
>> +    description:
>> +      Should contain a phandle to an FPGA Manager.  Child FPGA Regions
>> +      inherit this property from their ancestor regions.  An fpga-mgr property
>> +      in a region will override any inherited FPGA manager.
>> +
>> +  partial-fpga-config:
>> +    type: boolean
>> +    description:
>> +      Set if partial reconfiguration is to be done, otherwise full
>> +      reconfiguration is done.
>> +
>> +  region-freeze-timeout-us:
>> +    description:
>> +      The maximum time in microseconds to wait for bridges to successfully
>> +      become disabled before the region has been programmed.
>> +
>> +  region-unfreeze-timeout-us:
>> +    description:
>> +      The maximum time in microseconds to wait for bridges to successfully
>> +      become enabled after the region has been programmed.
>> +
>> +required:
>> +  - compatible
>> +  - "#address-cells"
>> +  - "#size-cells"
>> +  - fpga-mgr
>> +
>> +additionalProperties: true
> 
> Why? This should only be used if another schema is going to include this
> one. That's not the case here.

In v2 we discussed this with Krzysztof. I used pattern properties from 
simple-bus.yaml in v2.

https://lore.kernel.org/all/b2dd8bcd-1e23-4b68-b7b7-c01b034fc1fe@linaro.org/

> 
> 'type: object' would be acceptable here as that says only nodes can be
> added.

What do you think should be patternProperties in this case?

I played with it a little bit but all nodes with @ are likely object type.
But what to do with others?
There are nodes as you see in examples like fixed-factor-clock nodes which are 
also object type.

Standard property like firmware-name or encrypted-fpga-config are coming via 
overlay for sure. Others are not permitted. That's why others then listed 
properties likely must be type object. Is there an elegant way to encode it?

> 
>> +
>> +examples:
>> +  - |
>> +    /*
>> +     * Full Reconfiguration without Bridges with DT overlay
>> +     */
>> +    fpga_region0: fpga-region {
> 
> Drop unused labels.

Actually it is kind of used. I kept it to be able to reference something in 
comment below.

> 
>> +      compatible = "fpga-region";
>> +      #address-cells = <1>;
>> +      #size-cells = <1>;
>> +      fpga-mgr = <&fpga_mgr0>;
>> +
>> +      /* DT Overlay contains: &fpga_region0 */
>> +      firmware-name = "zynq-gpio.bin";
>> +      gpio@...00000 {
>> +        compatible = "xlnx,xps-gpio-1.00.a";
>> +        reg = <0x40000000 0x10000>;
> 
> This example implies this is a translatable address, but the lack of
> 'ranges' in the parent prevents that. In turn, that means unit-addresses
> should be accepted in the parent node name as well.

v1 contains ranges property and it was removed based on Krzysztof comment that 
fpga-region has no unit address that's why ranges shouldn't be used.

https://lore.kernel.org/all/c3c92468-a1db-498b-b4a2-7926b84cb5ea@linaro.org/

Thanks,
Michal

Powered by blists - more mailing lists