Date:   Mon, 29 Aug 2016 15:25:17 +0000
From:   Roy Zang <roy.zang@....com>
To:     Kishon Vijay Abraham I <kishon@...com>,
        Mingkai Hu <mingkai.hu@....com>,
        Joao Pinto <Joao.Pinto@...opsys.com>,
        "bhelgaas@...gle.com" <bhelgaas@...gle.com>,
        "linux-pci@...r.kernel.org" <linux-pci@...r.kernel.org>,
        "arnd@...db.de" <arnd@...db.de>, Jingoo Han <jingoohan1@...il.com>,
        Pratyush Anand <pratyush.anand@...il.com>
CC:     Ley Foon Tan <lftan@...era.com>, Rob Herring <robh@...nel.org>,
        "Tanmay Inamdar" <tinamdar@....com>,
        Roy Zang <tie-fei.zang@...escale.com>,
        "Mingkai Hu" <mingkai.hu@...escale.com>,
        Minghuan Lian <minghuan.Lian@...escale.com>,
        Richard Zhu <Richard.Zhu@...escale.com>,
        Lucas Stach <l.stach@...gutronix.de>,
        Murali Karicheri <m-karicheri2@...com>,
        "Thomas Petazzoni" <thomas.petazzoni@...e-electrons.com>,
        Jason Cooper <jason@...edaemon.net>,
        Thierry Reding <thierry.reding@...il.com>,
        "Simon Horman" <horms@...ge.net.au>,
        Zhou Wang <wangzhou1@...ilicon.com>,
        "Gabriele Paoloni" <gabriele.paoloni@...wei.com>,
        Stanimir Varbanov <svarbanov@...sol.com>,
        David Daney <david.daney@...ium.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-arm-kernel@...ts.infradead.org" 
        <linux-arm-kernel@...ts.infradead.org>,
        "linux-omap@...r.kernel.org" <linux-omap@...r.kernel.org>,
        Carlos Palminha <CARLOS.PALMINHA@...opsys.com>
Subject: Re: Support for configurable PCIe endpoint

On 08/18/2016 07:25 AM, Kishon Vijay Abraham I wrote:
> Hi,
>
> On Wednesday 17 August 2016 03:19 PM, Mingkai Hu wrote:
>>
>>> -----Original Message-----
>>> From: Kishon Vijay Abraham I [mailto:kishon@...com]
>>> Sent: Thursday, August 04, 2016 6:02 PM
>>> To: Joao Pinto <Joao.Pinto@...opsys.com>; bhelgaas@...gle.com; linux-
>>> pci@...r.kernel.org; arnd@...db.de; Jingoo Han <jingoohan1@...il.com>;
>>> Pratyush Anand <pratyush.anand@...il.com>
>>> Cc: Ley Foon Tan <lftan@...era.com>; Rob Herring <robh@...nel.org>;
>>> Tanmay Inamdar <tinamdar@....com>; Roy Zang <tie-
>>> fei.zang@...escale.com>; Mingkai Hu <mingkai.hu@...escale.com>;
>>> Minghuan Lian <minghuan.Lian@...escale.com>; Richard Zhu
>>> <Richard.Zhu@...escale.com>; Lucas Stach <l.stach@...gutronix.de>;
>>> Murali Karicheri <m-karicheri2@...com>; Thomas Petazzoni
>>> <thomas.petazzoni@...e-electrons.com>; Jason Cooper
>>> <jason@...edaemon.net>; Thierry Reding <thierry.reding@...il.com>;
>>> Simon Horman <horms@...ge.net.au>; Zhou Wang
>>> <wangzhou1@...ilicon.com>; Gabriele Paoloni
>>> <gabriele.paoloni@...wei.com>; Stanimir Varbanov <svarbanov@mm-
>>> sol.com>; David Daney <david.daney@...ium.com>; linux-
>>> kernel@...r.kernel.org; linux-arm-kernel@...ts.infradead.org; linux-
>>> omap@...r.kernel.org; Carlos Palminha
>>> <CARLOS.PALMINHA@...opsys.com>
>>> Subject: Re: Support for configurable PCIe endpoint
>>>
>>> Hi,
>>>
>>> On Wednesday 03 August 2016 07:09 PM, Joao Pinto wrote:
>>>> Hi Kishon,
>>>>
>>>> On 8/3/2016 7:03 AM, Kishon Vijay Abraham I wrote:
>>>>> Hi,
>>>>>
>>>>> The PCIe controller present in TI's DRA7 SoC is capable of operating
>>>>> either in Root Complex mode or Endpoint mode. (It uses Synopsys
>>>>> Designware Core). I'd assume most of the PCIe controllers on other
>>>>> platforms that use Designware core should also be capable to operate
>>>>> in endpoint mode. But linux kernel right now supports only RC mode.
>>>>>
>>>>> PCIe endpoint support discussion came up briefly before [1] but it
>>>>> was felt the practical use case will find firmware more suitable and
>>>>> endpoint support in kernel can be used only for validation or demo.
>>>>>
>>>>> *) Modify platform driver to support EP mode (in my case pci-dra7xx.c).
>>>>>
>>>>> *) dt binding specific to EP mode should be created.
>>>>>
>>>>> Once I complete the implementation and start posting RFC patches, a
>>>>> lot of these will become clear. But I want to check if this sounds
>>>>> okay to you guys before starting the implementation.
>>>>>
>>>>> Let me know if you have some other ideas too.
>>>>>
>>>>> Cheers
>>>>> Kishon
>>>>>
>>>>> [1] -> http://www.spinics.net/lists/linux-pci/msg26026.html
>>>>>
>>>> You are rising a topic that we are also addressing in Synopsys.
>>>>
>>>> For the PCIe RC hardware validation we are currently using the
>>>> standard pcie-designware and pcie-designware-plat drivers.
>>>>
>>>> For the Endpoint we have to use an internal software package. Its main
>>>> purpose is to initialize the IP registers, eDMA channels and make data
>>>> transfer to prove that the everything is working properly. This is
>>>> done in 2 levels, a custom driver built and loaded and an application
>>>> that makes some ioctl to the driver executing some interesting
>>>> functions to check the Endpoint status and make some data exchange.
>>> hmm.. the platform I have doesn't have a DMA in PCIe IP
>>> (http://www.ti.com/lit/ug/spruhz6g/spruhz6g.pdf). So in your testing does
>>> the EP access RC memory? i.e the driver in the RC allocates memory from it's
>>> DDR and gives it's DDR address to the EP. The EP then transfers data to this
>>> address. (This is a typical use case with ethernet PCIe cards). IIUC that's not
>>> simple with configurable EPs. I'd like to know more about your testing though.
>>>
>> Hi Kishon,
>>
>> This is a typical user case for EP to use DMA transfer data to/from RC memory.
>> In our case, we implement ring (like BD ring) or register in EP to communicate
>> The address allocated in RC memory, then EP can transfer data to/from RC memory.
> Initially I had some confusion w.r.t this because the address allocated in RC
> memory can also be an address in EP system. For example let's assume we connect
> two similar systems one configured as RC and the other configured as EP. The
> PCI driver in the RC allocates memory in it's DDR (say 0x80000000) and programs
> this address in the EP. Since it's a similar system, 0x80000000 will also be an
> address in the EPs DDR. This will result in EP transferring data to it's own
> DDR (at 0x80000000) instead of the same address in RC.
>
> But later realized instead of directly using the DDR address given by RC, this
> address should only be used to program the outbound window. That way the target
> of the outbound window can be an address given by the RC and source should be
> an address from the address space in the EP's system.
>
> Do you also use the RC memory address to program the outbound window?
>

When EP access RC memory, from EP perspective, there should be a offset
added to 0x800000000 to match the pcie outbound access  window.
Thanks.
Roy

Powered by blists - more mailing lists