linux-kernel - Re: [PATCH 0/7] Add a DRM driver to support AI Processing Unit (APU)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 24 May 2023 13:27:00 +0300
From:   Oded Gabbay <ogabbay@...nel.org>
To:     Kevin Hilman <khilman@...libre.com>
Cc:     Jeffrey Hugo <quic_jhugo@...cinc.com>,
        Alexandre Bailon <abailon@...libre.com>, airlied@...il.com,
        daniel@...ll.ch, maarten.lankhorst@...ux.intel.com,
        mripard@...nel.org, tzimmermann@...e.de,
        devicetree@...r.kernel.org, conor+dt@...nel.org, bero@...libre.com,
        jstephan@...libre.com, linux-kernel@...r.kernel.org,
        dri-devel@...ts.freedesktop.org, christian.koenig@....com,
        linaro-mm-sig@...ts.linaro.org, robh+dt@...nel.org,
        linux-mediatek@...ts.infradead.org, nbelin@...libre.com,
        krzysztof.kozlowski+dt@...aro.org, matthias.bgg@...il.com,
        linux-media@...r.kernel.org, sumit.semwal@...aro.org,
        linux-arm-kernel@...ts.infradead.org,
        angelogioacchino.delregno@...labora.com
Subject: Re: [PATCH 0/7] Add a DRM driver to support AI Processing Unit (APU)

On Wed, May 24, 2023 at 2:34 AM Kevin Hilman <khilman@...libre.com> wrote:
>
> Jeffrey Hugo <quic_jhugo@...cinc.com> writes:
>
> > On 5/17/2023 8:52 AM, Alexandre Bailon wrote:
> >> This adds a DRM driver that implements communication between the CPU and an
> >> APU. The driver target embedded device that usually run inference using some
> >> prebuilt models. The goal is to provide common infrastructure that could be
> >> re-used to support many accelerators. Both kernel, userspace and firmware tries
> >> to use standard and existing to leverage the development and maintenance effort.
> >> The series implements two platform drivers, one for simulation and another one for
> >> the mt8183 (compatible with mt8365).
> >
> > This looks like the 3 existing Accel drivers.  Why is this in DRM?
>
> Yes, this belongs in accel.  I think Alex had some issues around the
> infra in accel with device nodes not appearing/opening properly, but
> I'll let him comment there.  But either way, the right approach should
> be to fix any issues in accel and move it there.
>
> [...]
>
> >>   .../devicetree/bindings/gpu/mtk,apu-drm.yaml  |  38 ++
> >>   drivers/gpu/drm/Kconfig                       |   2 +
> >>   drivers/gpu/drm/Makefile                      |   1 +
> >>   drivers/gpu/drm/apu/Kconfig                   |  22 +
> >>   drivers/gpu/drm/apu/Makefile                  |  10 +
> >>   drivers/gpu/drm/apu/apu_drv.c                 | 282 +++++++++
> >>   drivers/gpu/drm/apu/apu_gem.c                 | 230 +++++++
> >>   drivers/gpu/drm/apu/apu_internal.h            | 205 ++++++
> >>   drivers/gpu/drm/apu/apu_sched.c               | 592 ++++++++++++++++++
> >>   drivers/gpu/drm/apu/simu_apu.c                | 313 +++++++++
> >>   include/uapi/drm/apu_drm.h                    |  81 +++
> >
> > "apu" seems too generic.  We already have 3 "AI processing units" over
> > in drivers/accel already...
>
> Indeed, it is generic, but that's kind of the point for this driver
> since it's targetted at generalizing the interface with "AI processing
> units" on a growing number of embedded SoCs (ARM, RISC-V, etc.)  In
> addition, the generic naming is intentional because the goal is bigger
> than the kernel and is working towards a generic, shared "libAPU"
> userspace[1], but also common firmware for DSP-style inference engines
> (e.g. analgous Sound Open Firmware for audio DSPs.)
>
> As usual, the various SoC vendors use different names (APU, NPU, NN
> unit, etc.)  but we'd like a generic name for the class of devices
> targetted by this driver.  And unfortunately, it looks like the equally
> generic "Versatile processing unit" is already taken Intel's
> drivers/accel/ivpu. :)
>
> Maybe since this is more about generalizing the interface between the
> CPU running linux and the APU, what about the name apu_if?  But I guess
> that applies to the other 3 drivers in drivers/accell also.  Hmmm...
>
> Naming things is hard[2], so we're definitly open to other ideas.  Any
> suggestions?
Maybe model it according to the tiny driver in drm display ? You can
then call it tiny_apu :-)
Disclosure: It was Daniel's suggestion, he can chime in with more
details on the tiny driver concept.
Oded

>
> Kevin
>
> [1] https://gitlab.baylibre.com/baylibre/libapu/libapu
>
> [2]
> "There are 2 hard problems in computer science: cache invalidation,
>  naming things and off-by-1 errors."
>  -- https://twitter.com/secretGeek/status/7269997868
>