linux-kernel - Re: [PATCH] venus: vdec: update output buffer size during vdec_s

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Wed, 25 Aug 2021 16:11:17 -0700
From:   Nathan Hebert <nhebert@...omium.org>
To:     Mansur Alisha Shaik <mansur@...eaurora.org>
Cc:     linux-media@...r.kernel.org, stanimir.varbanov@...aro.org,
        linux-kernel@...r.kernel.org, linux-arm-msm@...r.kernel.org,
        vgarodia@...eaurora.org, dikshita@...eaurora.org
Subject: Re: [PATCH] venus: vdec: update output buffer size during vdec_s_fmt()

Hi Mansur,

I tested the patch on kernel 5.4 on an Acer Chromebook Spin 513 based
on the Qualcomm Snapdragon 7c platform.

VP8 test vectors [0] were able to be decoded and decoded picture MD5's
matched the vpxdec reference decoder. Prior to this patch, three files
failed to decode. I didn't see any regressions with VP9 Profile 0
(8-bit) test vectors [1] using the same decode and checksum test
methods.

[0] https://chromium.googlesource.com/webm/vp8-test-vectors
[1] https://www.webmproject.org/vp9/levels/#test-bitstreams

Tested-by: Nathan Hebert <nhebert@...omium.org>

Best regards,
Nathan Hebert

On Wed, Aug 25, 2021 at 1:44 AM Mansur Alisha Shaik
<mansur@...eaurora.org> wrote:
>
> Video driver maintains an internal context for the output buffer size.
> During S_fmt() on capture plane, the output buffer size is not updated
> in driver context. As a result, during buf_prepare(), the size of the
> vb2_plane and internal size of the buffer, as maintained by the driver,
> does not match. This leads to buf_prepare() failure.
> Update the instance context for the output buffer size during s_fmt().
>
> Signed-off-by: Mansur Alisha Shaik <mansur@...eaurora.org>
> ---
>  drivers/media/platform/qcom/venus/vdec.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
> index 198e47eb63f4..c129b061a325 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -332,8 +332,11 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
>
>         if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>                 inst->fmt_out = fmt;
> -       else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> +       else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>                 inst->fmt_cap = fmt;
> +               inst->output2_buf_size =
> +                       venus_helper_get_framesz(pixfmt_cap, orig_pixmp.width, orig_pixmp.height);
> +       }
>
>         return 0;
>  }
> --
> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
> of Code Aurora Forum, hosted by The Linux Foundation
>