linux-kernel - Re: [RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <648d74af500d21e21204d998e65f9efeb2cea204.camel@ndufresne.ca>
Date:   Tue, 15 Nov 2022 11:19:51 -0500
From:   Nicolas Dufresne <nicolas@...fresne.ca>
To:     Tomasz Figa <tfiga@...omium.org>,
        Laurent Pinchart <laurent.pinchart@...asonboard.com>
Cc:     Hsia-Jun Li <Randy.Li@...aptics.com>, mchehab@...nel.org,
        hans.verkuil@...co.com, sakari.ailus@....fi,
        boris.brezillon@...labora.com, hiroh@...omium.org,
        Brian.Starkey@....com, kernel@...labora.com,
        narmstrong@...libre.com, linux-kernel@...r.kernel.org,
        frkoenig@...omium.org, stanimir.varbanov@...aro.org,
        Hans Verkuil <hverkuil@...all.nl>, linux-media@...r.kernel.org
Subject: Re: [RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify
 single/multi-planar handling (and more)

Le vendredi 11 novembre 2022 à 17:54 +0900, Tomasz Figa a écrit :
> > > I feel like this would be only useful in the MMAP mode. Looking at how
> > > the other UAPIs are evolving, things are going towards
> > > userspace-managed allocations, using, for example, DMA-buf heaps. I
> > > think we should follow the trend and keep the MMAP mode just at the
> > > same level of functionality as is today and focus on improvements and
> > > new functionality for the DMABUF mode.
> > 
> > I agree, but we will need an API to expose the memory constraints of the
> > device, or userspace won't be able to allocate memory compatible with
> > the hardware or driver requirements.
> 
> Yes, I fully agree and that's why I think we should rather focus our
> efforts in that direction rather than expanding the existing MMAP
> capabilities.

I was once told that MMAP was mandatory to support in v4l2 drivers. I'd like to
get some clarification on the subject for sure. We can't break compat, unless we
spin v4l3 here.

One thing that come to mind, is that its not true that a V4L2 driver can always
be importer only. For cards like Xplorer X1600P PCIe Accelerator [1] from Blaize
(they are using a modified, but still generic, Hantro Driver), the CODECs memory
should be allocated on the card for best performance. Only the driver is aware
that there is memory on that card, and so it must export the buffers.

In a DMABuf import only future, that basically means the driver must implement a
DMABuf HEAP driver, and to make this usable by generic software, the constraints
API need to support telling userspace that this Heap is to be used. This gap
easily extend to DRM, which have per driver API to allocate memory, and in some
cases these API must be used (they don't have heaps for on-card memory
allocation). When this is happening inside the GFX stack, it works very well,
but when you need to integrate this with some V4L2 driver, its not really
practical and requires something like minigbm/gralloc, which is non-generic.

[1] https://www.blaize.com/products/ai-edge-computing-platforms/