linux-kernel - Re: [PATCH] media: mediatek: vcodec: fix the error sizeimage for 10bit bitstream

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date: Thu, 04 Apr 2024 14:28:59 -0400
From: Nicolas Dufresne <nicolas.dufresne@...labora.com>
To: Yunfei Dong <yunfei.dong@...iatek.com>, Nícolas "F .
 R . A . Prado" <nfraprado@...labora.com>, Sebastian Fricke
 <sebastian.fricke@...labora.com>, Hans Verkuil <hverkuil-cisco@...all.nl>,
 AngeloGioacchino Del Regno <angelogioacchino.delregno@...labora.com>,
 Benjamin Gaignard <benjamin.gaignard@...labora.com>, Nathan Hebert
 <nhebert@...omium.org>
Cc: Hsin-Yi Wang <hsinyi@...omium.org>, Fritz Koenig
 <frkoenig@...omium.org>,  Daniel Vetter <daniel@...ll.ch>, Steve Cho
 <stevecho@...omium.org>, linux-media@...r.kernel.org, 
 devicetree@...r.kernel.org, linux-kernel@...r.kernel.org, 
 linux-arm-kernel@...ts.infradead.org, linux-mediatek@...ts.infradead.org, 
 Project_Global_Chrome_Upstream_Group@...iatek.com
Subject: Re: [PATCH] media: mediatek: vcodec: fix the error sizeimage for
 10bit bitstream

Hi,

Le mercredi 03 avril 2024 à 17:30 +0800, Yunfei Dong a écrit :
> The sizeimage of each plane are calculated the same way for 8bit and
> 10bit bitstream. Need to enlarge the sizeimage with simeimage*5/4 for
> 10bit bitstream when try and set fmt.

Can we stop adding more layers of custom code and port to v4l2-common helpers
please.

regards,
Nicolas

> 
> Fixes: 9d86be9bda6c ("media: mediatek: vcodec: Add driver to support 10bit")
> Signed-off-by: Yunfei Dong <yunfei.dong@...iatek.com>
> ---
>  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  | 47 ++++++++++++++-----
>  1 file changed, 34 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> index 9107707de6c4..45209894f1fe 100644
> --- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> +++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> @@ -259,6 +259,7 @@ static int vidioc_try_fmt(struct mtk_vcodec_dec_ctx *ctx, struct v4l2_format *f,
>  		pix_fmt_mp->num_planes = 1;
>  		pix_fmt_mp->plane_fmt[0].bytesperline = 0;
>  	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		unsigned int dram_y, dram_c, dram_y_10bit, dram_c_10bit;
>  		int tmp_w, tmp_h;
>  
>  		/*
> @@ -280,22 +281,42 @@ static int vidioc_try_fmt(struct mtk_vcodec_dec_ctx *ctx, struct v4l2_format *f,
>  		    (pix_fmt_mp->height + 64) <= frmsize->max_height)
>  			pix_fmt_mp->height += 64;
>  
> -		mtk_v4l2_vdec_dbg(0, ctx,
> -				  "before resize wxh=%dx%d, after resize wxh=%dx%d, sizeimage=%d",
> -				  tmp_w, tmp_h, pix_fmt_mp->width, pix_fmt_mp->height,
> -				  pix_fmt_mp->width * pix_fmt_mp->height);
> +		dram_y = pix_fmt_mp->width * pix_fmt_mp->height;
> +		dram_c = dram_y / 2;
> +
> +		dram_y_10bit = dram_y * 5 / 4;
> +		dram_c_10bit = dram_y_10bit / 2;
>  
>  		pix_fmt_mp->num_planes = fmt->num_planes;
> -		pix_fmt_mp->plane_fmt[0].sizeimage =
> -				pix_fmt_mp->width * pix_fmt_mp->height;
> -		pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
> -
> -		if (pix_fmt_mp->num_planes == 2) {
> -			pix_fmt_mp->plane_fmt[1].sizeimage =
> -				(pix_fmt_mp->width * pix_fmt_mp->height) / 2;
> -			pix_fmt_mp->plane_fmt[1].bytesperline =
> -				pix_fmt_mp->width;
> +		if (pix_fmt_mp->num_planes == 1) {
> +			if (ctx->is_10bit_bitstream) {
> +				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width * 5 / 4;
> +				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y_10bit + dram_c_10bit;
> +			} else {
> +				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
> +				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y + dram_c;
> +			}
> +		} else {
> +			if (ctx->is_10bit_bitstream) {
> +				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width * 5 / 4;
> +				pix_fmt_mp->plane_fmt[1].bytesperline = pix_fmt_mp->width * 5 / 4;
> +
> +				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y_10bit;
> +				pix_fmt_mp->plane_fmt[1].sizeimage = dram_c_10bit;
> +			} else {
> +				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
> +				pix_fmt_mp->plane_fmt[1].bytesperline = pix_fmt_mp->width;
> +
> +				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y;
> +				pix_fmt_mp->plane_fmt[1].sizeimage = dram_c;
> +			}
>  		}
> +
> +		mtk_v4l2_vdec_dbg(0, ctx,
> +				  "before resize:%dx%d, after resize:%dx%d, sizeimage=0x%x_0x%x",
> +				  tmp_w, tmp_h, pix_fmt_mp->width, pix_fmt_mp->height,
> +				  pix_fmt_mp->plane_fmt[0].sizeimage,
> +				  pix_fmt_mp->plane_fmt[1].sizeimage);
>  	}
>  
>  	pix_fmt_mp->flags = 0;