linux-kernel - Re: [PATCH v3 5/8] media: hantro: hevc: Allow to produce 10-bit frames

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3849b99e6d139b02c670c9e8d1ec306b24f6f821.camel@collabora.com>
Date:   Tue, 22 Jun 2021 10:26:42 -0300
From:   Ezequiel Garcia <ezequiel@...labora.com>
To:     Benjamin Gaignard <benjamin.gaignard@...labora.com>,
        hverkuil@...all.nl, p.zabel@...gutronix.de, mchehab@...nel.org,
        shawnguo@...nel.org, s.hauer@...gutronix.de, festevam@...il.com,
        gregkh@...uxfoundation.org, mripard@...nel.org,
        paul.kocialkowski@...tlin.com, wens@...e.org,
        jernej.skrabec@...l.net, emil.l.velikov@...il.com,
        andrzej.p@...labora.com, jc@...esim.co.uk,
        jernej.skrabec@...il.com, nicolas@...fresne.ca
Cc:     kernel@...gutronix.de, linux-imx@....com,
        linux-media@...r.kernel.org, linux-rockchip@...ts.infradead.org,
        linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 5/8] media: hantro: hevc: Allow to produce 10-bit
 frames

Hi Benjamin,

On Fri, 2021-06-18 at 15:15 +0200, Benjamin Gaignard wrote:
> If Hantro driver receive an 10-bit encoded bitstream allow it
> to produce 10-bit frames.
> Check that we are not try to produce 10-bit frames from a 8-bit
> encoded bitstream.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@...labora.com>
> ---
>  drivers/staging/media/hantro/hantro_drv.c      | 18 ++++++++++++++++++
>  .../staging/media/hantro/hantro_g2_hevc_dec.c  | 18 ++++++++++++++----
>  drivers/staging/media/hantro/hantro_hevc.c     |  2 +-
>  drivers/staging/media/hantro/imx8m_vpu_hw.c    |  4 ++++
>  4 files changed, 37 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
> index f6635ceb5111..b6373934734e 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -243,6 +243,16 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
>         return vb2_queue_init(dst_vq);
>  }
>  
> +static bool hantro_is_10bit_dst_format(struct hantro_ctx *ctx)
> +{
> +       switch (ctx->vpu_dst_fmt->fourcc) {
> +       case V4L2_PIX_FMT_P010:
> +               return true;
> +       default:
> +               return false;
> +       }
> +}
> +
>  static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
>  {
>         if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> @@ -259,6 +269,10 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
>                         return -EINVAL;
>         } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
>                 const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
> +               struct hantro_ctx *ctx;
> +
> +               ctx = container_of(ctrl->handler,
> +                                  struct hantro_ctx, ctrl_handler);
>  
>                 if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
>                         /* Luma and chroma bit depth mismatch */
> @@ -270,6 +284,10 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
>                 if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
>                         /* No scaling support */
>                         return -EINVAL;
> +               if (sps->bit_depth_luma_minus8 == 0 &&
> +                   hantro_is_10bit_dst_format(ctx)) {
> +                       return -EINVAL;

I had some more time to think about this,
and I recalled that this topic was already discussed
some time ago [1].

This approach won't work. You need to restrict the pixel
formats returned by TRY/G_FMT after the SPS is set with S_EXT_CTRL,
as per the specification.

https://www.spinics.net/lists/kernel/msg3576779.html

Thanks,
Ezequiel