linux-kernel - Re: Enabling peer to peer device transactions for PCIe devices

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAPcyv4hdMkXOxj9hUDpnftA7UTGDa498eBugdePp8EWr6S80gA@mail.gmail.com>
Date:   Mon, 5 Dec 2016 09:40:38 -0800
From:   Dan Williams <dan.j.williams@...el.com>
To:     Jason Gunthorpe <jgunthorpe@...idianresearch.com>
Cc:     Stephen Bates <sbates@...thlin.com>,
        Haggai Eran <haggaie@...lanox.com>,
        Logan Gunthorpe <logang@...tatee.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        "linux-nvdimm@...1.01.org" <linux-nvdimm@...1.01.org>,
        "christian.koenig@....com" <christian.koenig@....com>,
        "Suravee.Suthikulpanit@....com" <suravee.suthikulpanit@....com>,
        "John.Bridgman@....com" <john.bridgman@....com>,
        "Alexander.Deucher@....com" <alexander.deucher@....com>,
        "Linux-media@...r.kernel.org" <linux-media@...r.kernel.org>,
        "dri-devel@...ts.freedesktop.org" <dri-devel@...ts.freedesktop.org>,
        Max Gurtovoy <maxg@...lanox.com>,
        "linux-pci@...r.kernel.org" <linux-pci@...r.kernel.org>,
        "serguei.sagalovitch@....com" <serguei.sagalovitch@....com>,
        "Paul.Blinzer@....com" <paul.blinzer@....com>,
        "Felix.Kuehling@....com" <felix.kuehling@....com>,
        "ben.sander@....com" <ben.sander@....com>
Subject: Re: Enabling peer to peer device transactions for PCIe devices

On Mon, Dec 5, 2016 at 9:18 AM, Jason Gunthorpe
<jgunthorpe@...idianresearch.com> wrote:
> On Sun, Dec 04, 2016 at 07:23:00AM -0600, Stephen Bates wrote:
>> Hi All
>>
>> This has been a great thread (thanks to Alex for kicking it off) and I
>> wanted to jump in and maybe try and put some summary around the
>> discussion. I also wanted to propose we include this as a topic for LFS/MM
>> because I think we need more discussion on the best way to add this
>> functionality to the kernel.
>>
>> As far as I can tell the people looking for P2P support in the kernel fall
>> into two main camps:
>>
>> 1. Those who simply want to expose static BARs on PCIe devices that can be
>> used as the source/destination for DMAs from another PCIe device. This
>> group has no need for memory invalidation and are happy to use
>> physical/bus addresses and not virtual addresses.
>
> I didn't think there was much on this topic except for the CMB
> thing.. Even that is really a mapped kernel address..
>
>> I think something like the iopmem patches Logan and I submitted recently
>> come close to addressing use case 1. There are some issues around
>> routability but based on feedback to date that does not seem to be a
>> show-stopper for an initial inclusion.
>
> If it is kernel only with physical addresess we don't need a uAPI for
> it, so I'm not sure #1 is at all related to iopmem.
>
> Most people who want #1 probably can just mmap
> /sys/../pci/../resourceX to get a user handle to it, or pass around
> __iomem pointers in the kernel. This has been asked for before with
> RDMA.
>
> I'm still not really clear what iopmem is for, or why DAX should ever
> be involved in this..

Right, by default remap_pfn_range() does not establish DMA capable
mappings. You can think of iopmem as remap_pfn_range() converted to
use devm_memremap_pages(). Given the extra constraints of
devm_memremap_pages() it seems reasonable to have those DMA capable
mappings be optionally established via a separate driver.