linux-kernel - Re: [PATCH] media: venus: dec: Fix capture formats enumeration order

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 3 Mar 2023 15:09:18 -0700
From:   Jordan Crouse <jorcrous@...zon.com>
To:     Javier Martinez Canillas <javierm@...hat.com>
CC:     <linux-kernel@...r.kernel.org>, Albert Esteve <aesteve@...hat.com>,
        Stanimir Varbanov <stanimir.varbanov@...aro.org>,
        Sergio Lopez <slp@...hat.com>,
        Enric Balletbo i Serra <eballetb@...hat.com>,
        Andy Gross <agross@...nel.org>,
        Bjorn Andersson <andersson@...nel.org>,
        Konrad Dybcio <konrad.dybcio@...aro.org>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Stanimir Varbanov <stanimir.k.varbanov@...il.com>,
        Vikash Garodia <quic_vgarodia@...cinc.com>,
        <linux-arm-msm@...r.kernel.org>, <linux-media@...r.kernel.org>
Subject: Re: [PATCH] media: venus: dec: Fix capture formats enumeration order

On Fri, Feb 10, 2023 at 09:18:35AM +0100, Javier Martinez Canillas wrote:
> Commit 9593126dae3e ("media: venus: Add a handling of QC08C compressed
> format") and commit cef92b14e653 ("media: venus: Add a handling of QC10C
> compressed format") added support for the QC08C and QC10C compressed
> formats respectively.
> 
> But these also caused a regression, because the new formats where added
> at the beginning of the vdec_formats[] array and the vdec_inst_init()
> function sets the default format output and capture using fixed indexes
> of that array:
> 
> static void vdec_inst_init(struct venus_inst *inst)
> {
> ...
> 	inst->fmt_out = &vdec_formats[8];
> 	inst->fmt_cap = &vdec_formats[0];
> ...
> }
> 
> Since now V4L2_PIX_FMT_NV12 is not the first entry in the array anymore,
> the default capture format is not set to that as it was done before.
> 
> Both commits changed the first index to keep inst->fmt_out default format
> set to V4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out
> default format set to V4L2_PIX_FMT_NV12.
> 
> Rather than updating the index to the current V4L2_PIX_FMT_NV12 position,
> let's reorder the entries so that this format is the first entry again.
> 
> This would also make VIDIOC_ENUM_FMT report the V4L2_PIX_FMT_NV12 format
> with an index 0 as it did before the QC08C and QC10C formats were added.
> 
> Fixes: 9593126dae3e ("media: venus: Add a handling of QC08C compressed format")
> Fixes: cef92b14e653 ("media: venus: Add a handling of QC10C compressed format")
> Signed-off-by: Javier Martinez Canillas <javierm@...hat.com>

I just came across this issue independently and can confirm this patch fixes
the GStreamer V4L2 decoder on QRB5165.

Tested-by: Jordan Crouse <jorcrous@...zon.com>

> ---
> 
>  drivers/media/platform/qcom/venus/vdec.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
> index 4ceaba37e2e5..bb14bea9fe09 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -31,15 +31,15 @@
>   */
>  static const struct venus_format vdec_formats[] = {
>  	{
> -		.pixfmt = V4L2_PIX_FMT_QC08C,
> +		.pixfmt = V4L2_PIX_FMT_NV12,
>  		.num_planes = 1,
>  		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>  	}, {
> -		.pixfmt = V4L2_PIX_FMT_QC10C,
> +		.pixfmt = V4L2_PIX_FMT_QC08C,
>  		.num_planes = 1,
>  		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> -	},{
> -		.pixfmt = V4L2_PIX_FMT_NV12,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_QC10C,
>  		.num_planes = 1,
>  		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>  	}, {