linux-kernel - Re: [PATCH v3 00/14] Driver of Intel(R) Gaussian & Neural Accelerator

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <3aac3e39-4889-22dc-83dc-72fff63cb3d0@suse.de>
Date:   Mon, 17 May 2021 21:49:49 +0200
From:   Thomas Zimmermann <tzimmermann@...e.de>
To:     Alex Deucher <alexdeucher@...il.com>
Cc:     Arnd Bergmann <arnd@...db.de>, Jonathan Corbet <corbet@....net>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Dragan Cvetic <dragan.cvetic@...inx.com>,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        DRI Development <dri-devel@...ts.freedesktop.org>,
        Maciej Kwapulinski <maciej.kwapulinski@...ux.intel.com>,
        Andy Shevchenko <andy.shevchenko@...il.com>,
        Derek Kiernan <derek.kiernan@...inx.com>
Subject: Re: [PATCH v3 00/14] Driver of Intel(R) Gaussian & Neural Accelerator

Hi

Am 17.05.21 um 21:23 schrieb Alex Deucher:
> On Mon, May 17, 2021 at 3:12 PM Thomas Zimmermann <tzimmermann@...e.de> 
wrote:
>>
>> Hi
>>
>> Am 17.05.21 um 09:40 schrieb Daniel Vetter:
>>> On Fri, May 14, 2021 at 11:00:38AM +0200, Arnd Bergmann wrote:
>>>> On Fri, May 14, 2021 at 10:34 AM Greg Kroah-Hartman
>>>> <gregkh@...uxfoundation.org> wrote:
>>>>> On Thu, May 13, 2021 at 01:00:26PM +0200, Maciej Kwapulinski wrote:
>>>>>> Dear kernel maintainers,
>>>>>>
>>>>>> This submission is a kernel driver to support Intel(R) Gaussian & Neural
>>>>>> Accelerator (Intel(R) GNA). Intel(R) GNA is a PCI-based neural co-processor
>>>>>> available on multiple Intel platforms. AI developers and users can 
offload
>>>>>> continuous inference workloads to an Intel(R) GNA device in order to
>> free
>>>>>> processor resources and save power. Noise reduction and speech recognition
>>>>>> are the examples of the workloads Intel(R) GNA deals with while its usage
>>>>>> is not limited to the two.
>>>>>
>>>>> How does this compare with the "nnpi" driver being proposed here:
>>>>>           https://lore.kernel.org/r/20210513085725.45528-1-guy.zadicario@intel.com
>>>>>
>>>>> Please work with those developers to share code and userspace api and
>>>>> tools.  Having the community review two totally different apis and
>>>>> drivers for the same type of functionality from the same company is
>>>>> totally wasteful of our time and energy.
>>>>
>>>> Agreed, but I think we should go further than this and work towards a
>>>> subsystem across companies for machine learning and neural networks
>>>> accelerators for both inferencing and training.
>>>
>>> We have, it's called drivers/gpu. Feel free to rename to drivers/xpu or
>>> think G as in General, not Graphisc.
>>
>> I hope this was a joke.
>>
>> Just some thoughts:
>>
>> AFAICT AI first came as an application of GPUs, but has now
>> evolved/specialized into something of its own. I can imagine sharing
>> some code among the various subsystems, say GEM/TTM internals for memory
>> management. Besides that there's probably little that can be shared in
>> the userspace interfaces. A GPU is device that puts an image onto the
>> screen and an AI accelerator isn't. Treating both as the same, even if
>> they share similar chip architectures, seems like a stretch. They might
>> evolve in different directions and fit less and less under the same
>> umbrella.
> 
> The putting something on the screen is just a tiny part of what GPUs
> do these days.  Many GPUs don't even have display hardware anymore.
> Even with drawing APIs, it's just some operation that you do with
> memory.  The display may be another device entirely.  GPUs also do
> video encode and decode, jpeg acceleration, etc.  drivers/gpu seems
> like a logical place to me.  Call it drivers/accelerators if you like.
> Other than modesetting most of the shared infrastructure in
> drivers/gpu is around memory management and synchronization which are
> all the hard parts.  Better to try and share that than to reinvent
> that in some other subsystem.

I'm not sure whether we're on the same page or not.

I look at this from the UAPI perspective: the only interfaces that we 
really standardize among GPUs is modesetting, dumb buffers, GEM. The 
sophisticated rendering is done with per-driver interfaces. And 
modesetting is the thing that AI does not do.

Sharing common code among subsystems is not a problem. Many of our 
more-sophisticated helpers are located in DRM because no other 
subsystems have the requirements yet. Maybe AI now has and we can move 
the rsp shareable code to a common location. But AI is still no GPU. To 
give a bad analogy: GPUs transmit audio these days. Yet we don't treat 
them as sound cards.


Best regards
Thomas

> 
> Alex
> 
>>
>> And as Dave mentioned, these devices are hard to obtain. We don't really
>> know what we sign up for.
>>
>> Just my 2 cents.
>>
>> Best regards
>> Thomas
>>
>>
>>
>> --
>> Thomas Zimmermann
>> Graphics Driver Developer
>> SUSE Software Solutions Germany GmbH
>> Maxfeldstr. 5, 90409 Nürnberg, Germany
>> (HRB 36809, AG Nürnberg)
>> Geschäftsführer: Felix Imendörffer
>>

-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer



Download attachment "OpenPGP_signature" of type "application/pgp-signature" (841 bytes)