linux-kernel - [PATCH v2] media: videobuf2: add V4L2_BUF_FLAG_HEADERS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-Id: <20221125020741.28239-1-ming.qian@nxp.com>
Date:   Fri, 25 Nov 2022 10:07:41 +0800
From:   Ming Qian <ming.qian@....com>
To:     mchehab@...nel.org, hverkuil-cisco@...all.nl
Cc:     nicolas@...fresne.ca, tfiga@...omium.org, shawnguo@...nel.org,
        robh+dt@...nel.org, s.hauer@...gutronix.de, kernel@...gutronix.de,
        festevam@...il.com, linux-imx@....com, xiahong.bao@....com,
        linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-arm-kernel@...ts.infradead.org
Subject: [PATCH v2] media: videobuf2: add V4L2_BUF_FLAG_HEADERS_ONLY flag

By setting the V4L2_BUF_FLAG_HEADERS_ONLY flag,
hint the vb2 only contains stream header,
but does not contain any frame data.

This flag needs to be used when header mode is set to
V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE.

For decoder, there are some scenarios user only
queue the header in a separate buffer,
especially in android case,
but some vpu requires one buffer contains one frame,
for example, the amphion vpu, if the buffer only contains header,
the driver will merge the header to the next IDR frame.
So driver need this flag to tell this case.

For encoder, it's an enhancement.
when V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE is set,
encoder can return vps, sps, pps separately.

Signed-off-by: Ming Qian <ming.qian@....com>
---
 Documentation/userspace-api/media/v4l/buffer.rst     | 12 ++++++++++++
 .../userspace-api/media/v4l/ext-ctrls-codec.rst      | 10 +++++++---
 include/uapi/linux/videodev2.h                       |  2 ++
 3 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst
index 4638ec64db00..f00521c7ec48 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst
@@ -607,6 +607,18 @@ Buffer Flags
 	the format. Any subsequent call to the
 	:ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl will not block anymore,
 	but return an ``EPIPE`` error code.
+    * .. _`V4L2-BUF-FLAG-HEADERS-ONLY`:
+
+      - ``V4L2_BUF_FLAG_HEADERS_ONLY``
+      - 0x00200000
+      - This flag may be set by user for decoder and by driver for encoder
+	when the buffer only contains the codec header,
+	but does not contain any frame data. Usually the codec
+	header is merged with the next IDR frame, with the flag
+	``V4L2_BUF_FLAG_KEYFRAME``, but there are still some scenarios that will
+	queue the header in a separate buffer. This flag can be set only when
+	the codec supports V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE,
+	and the header mode is set to V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE
     * .. _`V4L2-BUF-FLAG-REQUEST-FD`:
 
       - ``V4L2_BUF_FLAG_REQUEST_FD``
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 2a165ae063fb..e42d6b4b15f5 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1386,8 +1386,12 @@ enum v4l2_mpeg_video_intra_refresh_period_type -
     (enum)
 
 enum v4l2_mpeg_video_header_mode -
-    Determines whether the header is returned as the first buffer or is
-    it returned together with the first frame. Applicable to encoders.
+    Determines whether the header is returned as the first buffer
+    with flag V4L2_BUF_FLAG_HEADERS_ONLY or if
+    it is returned together with the first frame.
+    Applicable to encoders and decoders.
+    If it's not implemented in a driver,
+    V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME is to be assumed,
     Possible values are:
 
 .. raw:: latex
@@ -1401,7 +1405,7 @@ enum v4l2_mpeg_video_header_mode -
     :stub-columns: 0
 
     * - ``V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE``
-      - The stream header is returned separately in the first buffer.
+      - The stream header is returned separately in the first buffer with the flag V4L2_BUF_FLAG_HEADERS_ONLY.
     * - ``V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME``
       - The stream header is returned together with the first encoded
 	frame.
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 29da1f4b4578..8eba081a6674 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1156,6 +1156,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv)
 #define V4L2_BUF_FLAG_TSTAMP_SRC_SOE		0x00010000
 /* mem2mem encoder/decoder */
 #define V4L2_BUF_FLAG_LAST			0x00100000
+/* Buffer only contains the codec header */
+#define V4L2_BUF_FLAG_HEADERS_ONLY		0x00200000
 /* request_fd is valid */
 #define V4L2_BUF_FLAG_REQUEST_FD		0x00800000
 
-- 
2.38.1