linux-kernel - Re: [PATCH 3/4] v4l2-ctrl: Add control for intra only decode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <a0ee566a587c28ffce97224abfa901520af5c83f.camel@ndufresne.ca>
Date:   Fri, 19 Jun 2020 08:39:07 -0400
From:   Nicolas Dufresne <nicolas@...fresne.ca>
To:     Stanimir Varbanov <stanimir.varbanov@...aro.org>,
        linux-media@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-arm-msm@...r.kernel.org
Cc:     Hans Verkuil <hverkuil-cisco@...all.nl>,
        Ezequiel Garcia <ezequiel@...labora.com>,
        Maheshwar Ajja <majja@...eaurora.org>,
        Mauro Carvalho Chehab <mchehab@...nel.org>
Subject: Re: [PATCH 3/4] v4l2-ctrl: Add control for intra only decode

Le mardi 16 juin 2020 à 15:30 +0300, Stanimir Varbanov a écrit :
> This adds a new decoder control to instruct the decoders to
> produce on its output intra frames only. Usually in this mode
> decoders might lower the count of output decoder buffers and
> hence reduce memory usage.

Perhaps I missed some discussion, would be nice if you could remind the
rationale from going away from a SKIP_MODE menu to adding dedicated boolean
control for each mode.

> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@...aro.org>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-codec.rst          | 9 +++++++++
>  drivers/media/v4l2-core/v4l2-ctrls.c                     | 2 ++
>  include/uapi/linux/v4l2-controls.h                       | 1 +
>  3 files changed, 12 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index b9d3f7ae6486..d7f34596f95b 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -652,6 +652,15 @@ enum v4l2_mpeg_video_bitrate_mode -
>      otherwise the decoder expects a single frame in per buffer.
>      Applicable to the decoder, all codecs.
>  
> +``V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY (boolean)``
> +    If enabled the decoder should start decoding only intra frames. The
> +    decoder consume first input buffer for progressive stream (or first
> +    two buffers for interlace). Decoder might not allocate more output
> +    buffers than it is required to consume one input frame. Usually the
> +    decoder input buffers will contain only intra frames but it is not
> +    mandatory. This control could be used for thumbnails generation.
> +    Applicable to the decoder, all codecs.

This imply that number of allocated buffers might be smaller (no references
buffer are needed), but I think it should actually be more explicit that this
must be set prior to reading MIN_BUFFER* and/or allocating buffers (since it's
userspace that allocates buffers).

What if a HW support live switching of this mode on key frames ? And if so, how
do we configure and control that ?

> +
>  ``V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE (boolean)``
>      Enable writing sample aspect ratio in the Video Usability
>      Information. Applicable to the H264 encoder.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-
> core/v4l2-ctrls.c
> index bc00d02e411f..2b1fb8dcd360 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -846,6 +846,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE:			return "H264
> MB Level Rate Control";
>  	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:			return
> "Sequence Header Mode";
>  	case V4L2_CID_MPEG_VIDEO_MAX_REF_PIC:			return "Max
> Number of Reference Pics";
> +	case V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY:	return "Decode
> intra frames only";
>  	case V4L2_CID_MPEG_VIDEO_H263_I_FRAME_QP:		return "H263 I-Frame
> QP Value";
>  	case V4L2_CID_MPEG_VIDEO_H263_P_FRAME_QP:		return "H263 P-Frame
> QP Value";
>  	case V4L2_CID_MPEG_VIDEO_H263_B_FRAME_QP:		return "H263 B-Frame
> QP Value";
> @@ -1197,6 +1198,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
> v4l2_ctrl_type *type,
>  	case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE:
>  	case V4L2_CID_MPEG_VIDEO_MPEG4_QPEL:
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:
> +	case V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY:
>  	case V4L2_CID_WIDE_DYNAMIC_RANGE:
>  	case V4L2_CID_IMAGE_STABILIZATION:
>  	case V4L2_CID_RDS_RECEPTION:
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-
> controls.h
> index 0f7e4388dcce..c64471e64aa7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -744,6 +744,7 @@ enum v4l2_cid_mpeg_video_hevc_size_of_length_field {
>  #define V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES	(V4L2_CID_MPEG_BASE +
> 643)
>  #define V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR	(V4L2_CID_MPEG_BASE +
> 644)
>  #define V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY		(V4L2_CID_MPEG_BASE +
> 645)
> +#define V4L2_CID_MPEG_VIDEO_DECODE_INTRA_FRAMES_ONLY	(V4L2_CID_MPEG_BASE +
> 646)
>  
>  /*  MPEG-class control IDs specific to the CX2341x driver as defined by V4L2
> */
>  #define V4L2_CID_MPEG_CX2341X_BASE				(V4L2_CTRL_CLASS
> _MPEG | 0x1000)