linux-kernel - Re: [RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <4F8FA564-9F66-4A12-8025-0B8D975A29E2@soulik.info>
Date:   Wed, 16 Nov 2022 20:38:25 +0800
From:   ayaka <ayaka@...lik.info>
To:     Nicolas Dufresne <nicolas@...fresne.ca>
Cc:     Tomasz Figa <tfiga@...omium.org>,
        Hsia-Jun Li <randy.li@...aptics.com>, mchehab@...nel.org,
        hans.verkuil@...co.com, laurent.pinchart@...asonboard.com,
        sakari.ailus@....fi, boris.brezillon@...labora.com,
        hiroh@...omium.org, Brian.Starkey@....com, kernel@...labora.com,
        narmstrong@...libre.com, linux-kernel@...r.kernel.org,
        frkoenig@...omium.org, stanimir.varbanov@...aro.org,
        Hans Verkuil <hverkuil@...all.nl>, linux-media@...r.kernel.org
Subject: Re: [RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)


> On Nov 16, 2022, at 12:03 AM, Nicolas Dufresne <nicolas@...fresne.ca> wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> Le vendredi 11 novembre 2022 à 17:52 +0900, Tomasz Figa a écrit :
>>>> Memory allocation doesn't sound to me like it is related to formats or
>>>> modifiers in any way. I agree with Nicolas that if we want to allow
>>>> the userspace to specify if the memory should be contiguous or not,
>>>> that should be a separate flag and actually I'd probably see it in
>>>> REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format.
>>> I agree with that. But here is a problem, if there was a display
>>> device(DRM) that only supports contiguous planes in a frame buffer.
>>> How do we be aware of that?
>> 
>> That's why I think the MMAP mode is not scalable and shouldn't be
>> expanded anymore. Both V4L2 and DRM devices should describe their
>> constraints to the userspace and then the userspace should allocate
>> accordingly from the right DMA-buf heap. (Or as Android and ChromeOS
>> do, just have a central allocator library that understands the
>> constraints, so there is no need to query the drivers.)
> 
> Just pointing out, they "hardcode" the constraints, they don't truly understand
> them. Also, the Android/ChromeOS implementation is not a great playground, as it
> completely ignores the constrait already exposed by V4L2 API (the sizeimage and
> bytesperline found in the FMT structure). You would not have to implement Hantro
> and Rockchip motion vector size calculation there if you'd simply use the
> sizeimage.
> 
About the allocation, I have my own plan here. Beyond the draft of DMAHeap allocator I threw last time.
I would like to modify the v4l2_queue API:
1. allowing different allocators for each memory plane(that matches the demand of synaptics video smart platform, which graphic planes support IOMMU while the compression metadata planes don’t).
2. New API that let driver allocate all memory planes for a buffer in a call

And DMAHeap V4l2 allocator would become a template or currying.

> Nicolas