linux-kernel - [PATCH v7 03/12] media: rkvdec: h264: Use bytesperline and buffer height as virstride

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20250225-rkvdec_h264_high10_and_422_support-v7-3-7992a68a4910@collabora.com>
Date: Tue, 25 Feb 2025 10:40:24 +0100
From: Sebastian Fricke <sebastian.fricke@...labora.com>
To: Mauro Carvalho Chehab <mchehab@...nel.org>, 
 Ezequiel Garcia <ezequiel@...guardiasur.com.ar>, 
 Greg Kroah-Hartman <gregkh@...uxfoundation.org>, 
 Hans Verkuil <hverkuil@...all.nl>, 
 Boris Brezillon <boris.brezillon@...labora.com>
Cc: linux-media@...r.kernel.org, linux-kernel@...r.kernel.org, 
 linux-rockchip@...ts.infradead.org, linux-staging@...ts.linux.dev, 
 Mauro Carvalho Chehab <mchehab+huawei@...nel.org>, 
 Alex Bee <knaerzche@...il.com>, 
 Nicolas Dufresne <nicolas.dufresne@...labora.com>, 
 Benjamin Gaignard <benjamin.gaignard@...labora.com>, 
 Detlev Casanova <detlev.casanova@...labora.com>, 
 Dan Carpenter <dan.carpenter@...aro.org>, Jonas Karlman <jonas@...boo.se>, 
 Christopher Obbard <christopher.obbard@...aro.org>
Subject: [PATCH v7 03/12] media: rkvdec: h264: Use bytesperline and buffer
 height as virstride

From: Jonas Karlman <jonas@...boo.se>

Use bytesperline and buffer height to calculate the strides configured.

This does not really change anything other than ensuring the
bytesperline that is signaled to userspace matches what is configured
in HW.

Signed-off-by: Jonas Karlman <jonas@...boo.se>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@...labora.com>
Tested-by: Nicolas Dufresne <nicolas.dufresne@...labora.com>
Tested-by: Christopher Obbard <chris.obbard@...labora.com>
---
 drivers/staging/media/rkvdec/rkvdec-h264.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 4fc167b42cf0c74fcc7adad5b867d9d7cd8af29b..7a1e76d423df55bd01c1ff26ff2c6d1b81ee7169 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -896,9 +896,9 @@ static void config_registers(struct rkvdec_ctx *ctx,
 	dma_addr_t rlc_addr;
 	dma_addr_t refer_addr;
 	u32 rlc_len;
-	u32 hor_virstride = 0;
-	u32 ver_virstride = 0;
-	u32 y_virstride = 0;
+	u32 hor_virstride;
+	u32 ver_virstride;
+	u32 y_virstride;
 	u32 yuv_virstride = 0;
 	u32 offset;
 	dma_addr_t dst_addr;
@@ -909,16 +909,16 @@ static void config_registers(struct rkvdec_ctx *ctx,
 
 	f = &ctx->decoded_fmt;
 	dst_fmt = &f->fmt.pix_mp;
-	hor_virstride = (sps->bit_depth_luma_minus8 + 8) * dst_fmt->width / 8;
-	ver_virstride = round_up(dst_fmt->height, 16);
+	hor_virstride = dst_fmt->plane_fmt[0].bytesperline;
+	ver_virstride = dst_fmt->height;
 	y_virstride = hor_virstride * ver_virstride;
 
 	if (sps->chroma_format_idc == 0)
 		yuv_virstride = y_virstride;
 	else if (sps->chroma_format_idc == 1)
-		yuv_virstride += y_virstride + y_virstride / 2;
+		yuv_virstride = y_virstride + y_virstride / 2;
 	else if (sps->chroma_format_idc == 2)
-		yuv_virstride += 2 * y_virstride;
+		yuv_virstride = 2 * y_virstride;
 
 	reg = RKVDEC_Y_HOR_VIRSTRIDE(hor_virstride / 16) |
 	      RKVDEC_UV_HOR_VIRSTRIDE(hor_virstride / 16) |

-- 
2.25.1