linux-kernel - RE: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <VI1PR04MB496066B27A378F1DA7223C9C92E70@VI1PR04MB4960.eurprd04.prod.outlook.com>
Date:   Thu, 12 Nov 2020 12:35:40 +0000
From:   Sherry Sun <sherry.sun@....com>
To:     Kishon Vijay Abraham I <kishon@...com>,
        Arnd Bergmann <arnd@...nel.org>
CC:     "bhelgaas@...gle.com" <bhelgaas@...gle.com>,
        Jonathan Corbet <corbet@....net>,
        "lorenzo.pieralisi@....com" <lorenzo.pieralisi@....com>,
        "arnd@...db.de" <arnd@...db.de>,
        "jdmason@...zu.us" <jdmason@...zu.us>,
        "dave.jiang@...el.com" <dave.jiang@...el.com>,
        "allenbh@...il.com" <allenbh@...il.com>,
        "tjoseph@...ence.com" <tjoseph@...ence.com>,
        Rob Herring <robh@...nel.org>,
        "gregkh@...uxfoundation.org" <gregkh@...uxfoundation.org>,
        "linux-pci@...r.kernel.org" <linux-pci@...r.kernel.org>,
        "linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-ntb@...glegroups.com" <linux-ntb@...glegroups.com>
Subject: RE: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express
 Non-Transparent Bridge

Hi Kishon,

> Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-
> Transparent Bridge
> 
> Hi Sherry,
> 
> On 11/11/20 8:19 am, Sherry Sun wrote:
> > Hi Kishon,
> >
> >> Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express
> >> Non- Transparent Bridge
> >>
> >> Hi Sherry, Arnd,
> >>
> >> On 10/11/20 8:29 pm, Arnd Bergmann wrote:
> >>> On Tue, Nov 10, 2020 at 3:20 PM Kishon Vijay Abraham I
> >>> <kishon@...com>
> >> wrote:
> >>>> On 10/11/20 7:55 am, Sherry Sun wrote:
> >>>
> >>>>> But for VOP, only two boards are needed(one board as host and one
> >>>>> board as card) to realize the communication between the two
> >>>>> systems,
> >> so my question is what are the advantages of using NTB?
> >>>>
> >>>> NTB is a bridge that facilitates communication between two
> >>>> different systems. So it by itself will not be source or sink of
> >>>> any data unlike a normal EP to RP system (or the VOP) which will be
> >>>> source or sink
> >> of data.
> >>>>
> >>>>> Because I think the architecture of NTB seems more complicated.
> >>>>> Many
> >> thanks!
> >>>>
> >>>> yeah, I think it enables a different use case all together.
> >>>> Consider you have two x86 HOST PCs (having RP) and they have to be
> >>>> communicate using PCIe. NTB can be used in such cases for the two
> >>>> x86 PCs to communicate with each other over PCIe, which wouldn't be
> >>>> possible
> >> without NTB.
> >>>
> >>> I think for VOP, we should have an abstraction that can work on
> >>> either NTB or directly on the endpoint framework but provide an
> >>> interface that then lets you create logical devices the same way.
> >>>
> >>> Doing VOP based on NTB plus the new NTB_EPF driver would also work
> >>> and just move the abstraction somewhere else, but I guess it would
> >>> complicate setting it up for those users that only care about the
> >>> simpler endpoint case.
> >>
> >> I'm not sure if you've got a chance to look at [1], where I added
> >> support for RP<->EP system both running Linux, with EP configured
> >> using Linux EP framework (as well as HOST ports connected to NTB
> >> switch, patches 20 and 21, that uses the Linux NTB framework) to
> >> communicate using virtio over PCIe.
> >>
> >
> > I saw your patches at [1], here you take a rpmsg as an example to
> > communicate between two SoCs using PCIe RC<->EP and HOST1-NTB-
> HOST2 for different usercases.
> > The VOP code works under the PCIe RC<->EP framework, which means that
> > we can also make VOP works under the Linux NTB framework, just like the
> rpmsg way you did here, right?
> 
> Does VOP really work with EP framework? At-least whatever is in upstream
> doesn't seem to indicate so.
> 

We did write a pci_epf driver to support VOP, looks like pci-epf-test.c, and it works well.
So certainly VOP can work with EP framework.
But it's a pity that the VOP related codes has been deleted before we send the pci_epf_vop driver patches to upstream.

> The NTB framework lets one host with RP port to communicate with another
> host with RP port.
> 
> The EP Framework lets one device with EP port to communicate with a host
> with RP port.
> 
> Rest of the trick should be how you tie them together.
> 
> PCIe framework creates "pci_device" for each of the devices it enumerates.
> NTB framework works on this pci_device to communicate with the remote
> host using PCIe bridge. The remote host will use NTB framework as well.
> 
> So depends on what interfaces VOP device provides you can use either NTB
> framework or EP framework. If it's going to connect two different devices in
> turn creating pci_device on each of the systems, then you can use NTB
> framework.
> 

Thanks for your detailed explanation! It is clear.
I think maybe VOP is suitable for the basic PCIe framework instead of NTB.

Best regards
Sherry

> Regards
> Kishon