linux-kernel - RE: [EXT] Re: [PATCH] media: videobuf2: add V4L2_BUF_FLAG

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Tue, 5 Jul 2022 11:34:44 +0000
From:   Ming Qian <ming.qian@....com>
To:     Nicolas Dufresne <nicolas@...fresne.ca>,
        "mchehab@...nel.org" <mchehab@...nel.org>,
        "hverkuil-cisco@...all.nl" <hverkuil-cisco@...all.nl>
CC:     "shawnguo@...nel.org" <shawnguo@...nel.org>,
        "robh+dt@...nel.org" <robh+dt@...nel.org>,
        "s.hauer@...gutronix.de" <s.hauer@...gutronix.de>,
        "kernel@...gutronix.de" <kernel@...gutronix.de>,
        "festevam@...il.com" <festevam@...il.com>,
        dl-linux-imx <linux-imx@....com>,
        "linux-media@...r.kernel.org" <linux-media@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-arm-kernel@...ts.infradead.org" 
        <linux-arm-kernel@...ts.infradead.org>
Subject: RE: [EXT] Re: [PATCH] media: videobuf2: add V4L2_BUF_FLAG_CODECCONFIG
 flag

>>From: Ming Qian
>>Sent: 2022年7月5日 9:52
>>To: Nicolas Dufresne <nicolas@...fresne.ca>; mchehab@...nel.org;
>>hverkuil-cisco@...all.nl
>>Cc: shawnguo@...nel.org; robh+dt@...nel.org; s.hauer@...gutronix.de;
>>kernel@...gutronix.de; festevam@...il.com; dl-linux-imx
>><linux-imx@....com>; linux-media@...r.kernel.org;
>>linux-kernel@...r.kernel.org; linux-arm-kernel@...ts.infradead.org
>>Subject: RE: [EXT] Re: [PATCH] media: videobuf2: add
>>V4L2_BUF_FLAG_CODECCONFIG flag
>>
>>> From: Nicolas Dufresne <nicolas@...fresne.ca>
>>> Sent: 2022年7月4日 23:53
>>> To: Ming Qian <ming.qian@....com>; mchehab@...nel.org;
>>> hverkuil-cisco@...all.nl
>>> Cc: shawnguo@...nel.org; robh+dt@...nel.org; s.hauer@...gutronix.de;
>>> kernel@...gutronix.de; festevam@...il.com; dl-linux-imx
>>> <linux-imx@....com>; linux-media@...r.kernel.org;
>>> linux-kernel@...r.kernel.org; linux-arm-kernel@...ts.infradead.org
>>> Subject: [EXT] Re: [PATCH] media: videobuf2: add
>>> V4L2_BUF_FLAG_CODECCONFIG flag
>>>
>>> Caution: EXT Email
>>>
>>> Le mardi 28 juin 2022 à 10:19 +0800, Ming Qian a écrit :
>>> > By setting the V4L2_BUF_FLAG_CODECCONFIG flag, user-space should be
>>> > able to hint decoder the vb2 only contains codec config header, but
>>> > does not contain any frame data.
>>> > It's only used for parsing header, and can't be decoded.
>>>
>>> This is copied from OMX specification. I think we we import this, we
>>> should at least refer to the original.
>>>
>>
>>Hi Nicolas,
>>    Do you mean OMX_BUFFERFLAG_CODECCONFIG?
>>    I'm sorry that I didn't notice it before.
>>    Currently we only encounter this requirement on Android, I'm not sure if
>>it has a reference to omx.
>>    And thank you very much for pointing out it.
>>
>
> Android media stack has been based on OMX for the last decade. They are slowly moving to CODEC2 which more or less is a similar abstraction with similar ideas. Let's research prior art, so we don't screw compatibility.
>

I got it, I'll try to study the android codec2, 
and do you agree that we should add V4L2_BUF_FLAG_CODECCONFIG flag, just like OMX_BUFFERFLAG_CODECCONFIG?
Or is there any other solution that can handle this case?

>>Ming
>>
>>> >
>>> > Current, it's usually used by android.
>>> >
>>> > Signed-off-by: Ming Qian <ming.qian@....com>
>>> > ---
>>> >  Documentation/userspace-api/media/v4l/buffer.rst | 9 +++++++++
>>> >  include/uapi/linux/videodev2.h                   | 2 ++
>>> >  2 files changed, 11 insertions(+)
>>> >
>>> > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
>>> > b/Documentation/userspace-api/media/v4l/buffer.rst
>>> > index 4638ec64db00..acdc4556f4f4 100644
>>> > --- a/Documentation/userspace-api/media/v4l/buffer.rst
>>> > +++ b/Documentation/userspace-api/media/v4l/buffer.rst
>>> > @@ -607,6 +607,15 @@ Buffer Flags
>>> >       the format. Any subsequent call to the
>>> >       :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl will not block anymore,
>>> >       but return an ``EPIPE`` error code.
>>> > +    * .. _`V4L2-BUF-FLAG-CODECCONFIG`:
>>> > +
>>> > +      - ``V4L2_BUF_FLAG_CODECCONFIG``
>>> > +      - 0x00200000
>>> > +      - This flag may be set when the buffer only contains codec config
>>> > +    header, but does not contain any frame data. Usually the codec
>>config
>>> > +    header is merged to the next idr frame, with the flag
>>> > +    ``V4L2_BUF_FLAG_KEYFRAME``, but there is still some scenes that
>>will
>>> > +    split the header and queue it separately.
>>>
>>> I think the documentation is clear. Now, if a driver uses this, will
>>> existing userland (perhaps good to check GStreamer, FFMPEG and
>>> Chromium ?) will break ?
>>> So we need existing driver to do this when flagged to, and just
>>> copy/append when the userland didn't opt-in that feature ?
>>>
>>> >      * .. _`V4L2-BUF-FLAG-REQUEST-FD`:
>>> >
>>> >        - ``V4L2_BUF_FLAG_REQUEST_FD`` diff --git
>>> > a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> > index 5311ac4fde35..8708ef257710
>>> > 100644
>>> > --- a/include/uapi/linux/videodev2.h
>>> > +++ b/include/uapi/linux/videodev2.h
>>> > @@ -1131,6 +1131,8 @@ static inline __u64 v4l2_timeval_to_ns(const
>>> struct timeval *tv)
>>> >  #define V4L2_BUF_FLAG_TSTAMP_SRC_SOE         0x00010000
>>> >  /* mem2mem encoder/decoder */
>>> >  #define V4L2_BUF_FLAG_LAST                   0x00100000
>>> > +/* Buffer only contains codec header */
>>> > +#define V4L2_BUF_FLAG_CODECCONFIG            0x00200000
>>> >  /* request_fd is valid */
>>> >  #define V4L2_BUF_FLAG_REQUEST_FD             0x00800000
>>> >