Message-ID: <DM6PR12MB30824C5129A7251C589F1461E8769@DM6PR12MB3082.namprd12.prod.outlook.com>
Date:   Mon, 29 Aug 2022 04:49:02 +0000
From:   "Gupta, Nipun" <Nipun.Gupta@....com>
To:     Robin Murphy <robin.murphy@....com>,
        Saravana Kannan <saravanak@...gle.com>,
        Jason Gunthorpe <jgg@...dia.com>
CC:     Greg KH <gregkh@...uxfoundation.org>,
        "robh+dt@...nel.org" <robh+dt@...nel.org>,
        "krzysztof.kozlowski+dt@...aro.org" 
        <krzysztof.kozlowski+dt@...aro.org>,
        "rafael@...nel.org" <rafael@...nel.org>,
        "eric.auger@...hat.com" <eric.auger@...hat.com>,
        "alex.williamson@...hat.com" <alex.williamson@...hat.com>,
        "cohuck@...hat.com" <cohuck@...hat.com>,
        "Gupta, Puneet (DCG-ENG)" <puneet.gupta@....com>,
        "song.bao.hua@...ilicon.com" <song.bao.hua@...ilicon.com>,
        "mchehab+huawei@...nel.org" <mchehab+huawei@...nel.org>,
        "maz@...nel.org" <maz@...nel.org>,
        "f.fainelli@...il.com" <f.fainelli@...il.com>,
        "jeffrey.l.hugo@...il.com" <jeffrey.l.hugo@...il.com>,
        "Michael.Srba@...nam.cz" <Michael.Srba@...nam.cz>,
        "mani@...nel.org" <mani@...nel.org>,
        "yishaih@...dia.com" <yishaih@...dia.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "devicetree@...r.kernel.org" <devicetree@...r.kernel.org>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "okaya@...nel.org" <okaya@...nel.org>,
        "Anand, Harpreet" <harpreet.anand@....com>,
        "Agarwal, Nikhil" <nikhil.agarwal@....com>,
        "Simek, Michal" <michal.simek@....com>,
        "git (AMD-Xilinx)" <git@....com>
Subject: RE: [RFC PATCH v2 2/6] bus/cdx: add the cdx bus driver

[AMD Official Use Only - General]



> -----Original Message-----
> From: Robin Murphy <robin.murphy@....com>
> Sent: Friday, August 26, 2022 1:28 AM
> To: Saravana Kannan <saravanak@...gle.com>; Jason Gunthorpe
> <jgg@...dia.com>
> Cc: Greg KH <gregkh@...uxfoundation.org>; Gupta, Nipun
> <Nipun.Gupta@....com>; robh+dt@...nel.org;
> krzysztof.kozlowski+dt@...aro.org; rafael@...nel.org; eric.auger@...hat.com;
> alex.williamson@...hat.com; cohuck@...hat.com; Gupta, Puneet (DCG-ENG)
> <puneet.gupta@....com>; song.bao.hua@...ilicon.com;
> mchehab+huawei@...nel.org; maz@...nel.org; f.fainelli@...il.com;
> jeffrey.l.hugo@...il.com; Michael.Srba@...nam.cz; mani@...nel.org;
> yishaih@...dia.com; linux-kernel@...r.kernel.org; devicetree@...r.kernel.org;
> kvm@...r.kernel.org; okaya@...nel.org; Anand, Harpreet
> <harpreet.anand@....com>; Agarwal, Nikhil <nikhil.agarwal@....com>;
> Simek, Michal <michal.simek@....com>; git (AMD-Xilinx) <git@....com>
> Subject: Re: [RFC PATCH v2 2/6] bus/cdx: add the cdx bus driver
> 
> [CAUTION: External Email]
> 
> On 2022-08-25 19:38, Saravana Kannan wrote:
> > On Wed, Aug 24, 2022 at 4:31 PM Jason Gunthorpe <jgg@...dia.com> wrote:
> >>
> >> On Wed, Aug 24, 2022 at 02:11:48PM +0200, Greg KH wrote:
> >>>> We can share the RFC in case you are interested in looking at code flow
> >>>> using the of_dynamic approach.
> >>>
> >>> Please no more abuse of the platform device.
> >>
> >> Last time this came up there was some disagreement from the ARM folks,
> >> they were not keen on having xx_drivers added all over the place to
> >> support the same OF/DT devices just discovered in a different way. It is
> >> why ACPI is mapped to platform_device even in some cases.
> >>
> >> I think if you push them down this path they will get resistance to
> >> get the needed additional xx_drivers into the needed places.
> >>
> >>> If your device can be discovered by scanning a bus, it is not a platform
> >>> device.
> >>
> >> A DT fragment loaded during boot binds a driver using a
> >> platform_driver, why should a DT fragment loaded post-boot bind using
> >> an XX_driver and further why should the CDX way of getting the DT
> >> raise to such importantance that it gets its own cdx_driver ?
> >>
> >> In the end the driver does not care about how the DT was loaded.
> >> None of these things are on a discoverable bus in any sense like PCI
> >> or otherwise. They are devices described by a DT fragement and they
> >> take all their parameters from that chunk of DT.
> >>
> >> How the DT was loaded into the system is not a useful distinction that
> >> raises the level of needing an entire new set of xx_driver structs all
> >> over the tree, IMHO.
> >
> > Jason, I see your point or rather the point the ARM folks might have
> > made. But in this case, why not use DT overlays to add these devices?
> > IIRC there's an in kernel API to add DT overlays. If so, should this
> > be more of a FPGA driver that reads FPGA stuff and adds DT overlays?
> > That'd at least make a stronger case for why this isn't a separate
> > bus.
> 
> Right, that's exactly where this discussion started.
> 
> To my mind, it would definitely help to understand if this is a *real*
> discoverable bus in hardware, i.e. does one have to configure one's
> device with some sort of CDX wrapper at FPGA synthesis time, that then
> physically communicates with some sort of CDX controller to identify
> itself once loaded; or is it "discoverable" in the sense that there's
> some firmware on an MCU controlling what gets loaded into the FPGA, and
> software can query that and get back whatever precompiled DTB fragment
> came bundled with the bitstream, i.e. it's really more like fpga-mgr in
> a fancy hat?

Devices are created in FPFGA with a CDX wrapper, and CDX controller(firmware)
reads that CDX wrapper to find out new devices. Host driver then interacts with
firmware to find newly discovered devices. This bus aligns with PCI infrastructure.
It happens to be an embedded interface as opposed to off-chip connection.

We are trying to do an RFC which proposes CDX as a new bus. It seems to be a
cleaner interface than what was added in RFC v2.

> 
> It's pretty much impossible to judge from all the empty placeholder code
> here how much is real and constrained by hardware and how much is
> firmware abstraction, which makes it particularly hard to review whether
> any proposal heading in the right direction.

You can consider the placeholders for now as API calls which would eventually
communicate with FW, and fetch required info like number of FPGA devices,
device related parameters (vendor_id, device_id etc), and command the
firmware to reset the device.

In next rev, we would add new API stubs instead of empty placeholders (as FW
interaction code is under development), which could give more clear view.

> 
> Even if it *is* entirely firmware smoke-and-mirrors, if that firmware
> can provide a standardised discovery and configuration interface for
> common resources, it can be a bus. But then it should *be* a bus, with
> its own bus_type and its own device type to model those standard
> interfaces and IDs and resources. Or if it is really just a very clever
> dynamic DT overlay manager for platform devices, then it can be that
> instead. But what it should clearly not be is some in-between mess
> making the worst of both worlds, which is what the code here inescapably
> smells of.
> 
> Thanks,
> Robin.

Powered by blists - more mailing lists