linux-kernel - Re: New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20171017134810.000049a9@huawei.com>
Date:   Tue, 17 Oct 2017 13:48:10 +0100
From:   Jonathan Cameron <Jonathan.Cameron@...wei.com>
To:     Andrew Donnellan <andrew.donnellan@....ibm.com>
CC:     LKML <linux-kernel@...r.kernel.org>, <jic23@...nel.org>,
        "Liguozhu (Kenneth)" <liguozhu@...ilicon.com>,
        Ilias Apalodimas <ilias.apalodimas@...aro.org>,
        <francois.ozog@...aro.org>, <Prasad.Athreya@...ium.com>,
        <arndbergmann@...il.com>,
        Alex Williamson <alex.williamson@...hat.com>,
        Frederic Barrat <fbarrat@...ux.vnet.ibm.com>,
        Mark Brown <broonie@...nel.org>,
        <Tirumalesh.Chalamarla@...ium.com>, <jcm@...hat.com>,
        Ard Biesheuvel <ard.biesheuvel@...aro.org>,
        "Jean-Philippe Brucker" <jean-philippe.brucker@....com>,
        Kirti Wankhede <kwankhede@...dia.com>,
        Eric Auger <eric.auger@...hat.com>, <kvm@...r.kernel.org>,
        <linux-crypto@...r.kernel.org>, <linuxarm@...wei.com>,
        Balbir Singh <bsingharora@...il.com>,
        Benjamin Herrenschmidt <benh@...nel.crashing.org>,
        Alistair Popple <alistair@...ple.id.au>,
        "Alastair D'Silva" <alastair@...ilva.org>,
        Gery Schneider <gery.schneider@...ibm.com>,
        Francois Richard <Francois.Richard@...ibm.com>,
        Jonathan Corbet <corbet@....net>
Subject: Re: New Linux accelerators discussion list [was: Re: Fostering
 linux community collaboration on hardware accelerators]

On Tue, 17 Oct 2017 11:00:40 +1100
Andrew Donnellan <andrew.donnellan@....ibm.com> wrote:

> On 17/10/17 01:07, Jonathan Cameron wrote:
> > <snip>
> >   
> >>> So as ever with a linux community focusing on a particular topic, the
> >>> obvious solution is a mailing list. There are a number of options on how
> >>> do this.
> >>>
> >>> 1) Ask one of the industry bodies to host? Who?
> >>>
> >>> 2) Put together a compelling argument for linux-accelerators@...r.kernel.org
> >>> as probably the most generic location for such a list.  
> >>
> >> Happy to offer linux-accelerators@...ts.ozlabs.org, which I can get set
> >> up immediately (and if we want patchwork, patchwork.ozlabs.org is
> >> available as always, no matter where the list is hosted).
> >>  
> > 
> > That would be great! Thanks for doing this. Much easier to find out what
> > such a list is useful for by the practical option of having a list and
> > see what people do with it.  
> 
> [+ LKML]
> 
> We now have a mailing list:
> 
>    List:      linux-accelerators@...ts.ozlabs.org
>    Info:      https://lists.ozlabs.org/listinfo/linux-accelerators
>    Archives:  https://lists.ozlabs.org/pipermail/linux-accelerators
> 
> I haven't set up Patchwork as yet, but if people think that's a good 
> idea I can get that done too.
> 
> 
> Andrew
> 

Thanks Andrew.

A quick summary of initial thoughts on scope of this list for anyone
entering the discussion at this point.
Note it will hopefully evolve in whatever direction people find helpful.
This contains some suggestions not a barrier to wider scope!

There are a number of projects / applications involving what are
termed hardware accelerators.  These include:
* Traditional offload engines
	- Crypto, compression, media transcoding and similar accelerators
	- usecases including kTLS, Storage Encryption etc.
* Dynamic FPGA type accelerators
* ODP, DPDK and similar networking data plane - particularly dual stack
  solutions where the kernel 'plays nicely' with userspace drivers.
* AI Accelerators
* Shared Virtual Memory (SVM) bus systems including Open-CAPI, CCIX etc
* Fast data flow to/from userspace applications.

A number of existing project focus on these:
* Mainline kernel drivers
	- Numerous crypto drivers etc
	- Open-CAPI
* Various networking data plane activities
* VFIO based and similar userspace drivers
Hopefully this list can provide a broader forum where more general
questions are being considered.

The discussion that lead to this list was that a number of people would
like a general open forum on which to discuss ideas with scope beyond
simply one kernel subsystem or one particular userspace framework.

Topics might include
* RFCs and early reviews of new approaches.
* Similar hardware - who is trying to solve the same problems?
* What would we ideally want from new hardware iterations?
* Hardware description - the question of how to chose a particular
  crypto engine is very dependent on the particular data flows.
  Sometimes hardware accelerators don't actually help due to overheads.
  Understanding those barriers would be very useful.
* Upstream paths - blockers and how to work with the communities to
  overcome them.
* Fostering stronger userspace communities to allow these accelerators to 
  be easily harnessed.
	- A number of projects have been highlighted in this thread
	  OpenStack (cyborg project), openMP accelerator support 
* Robustness / security of userspace frameworks.
* Virtualisation of accelerators

Anyhow, this email was just meant to draw together some thoughts.
It will be interesting to see what the list actually gets used for :)

Jonathan