linux-kernel - [PATCH] media: v4l: vsp1: fix and generalize offset calculation for plane cropping

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <1637679566-76975-1-git-send-email-mrodin@de.adit-jv.com>
Date:   Tue, 23 Nov 2021 15:59:26 +0100
From:   Michael Rodin <mrodin@...adit-jv.com>
To:     Laurent Pinchart <laurent.pinchart@...asonboard.com>,
        Kieran Bingham <kieran.bingham+renesas@...asonboard.com>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        <linux-media@...r.kernel.org>, <linux-renesas-soc@...r.kernel.org>,
        <linux-kernel@...r.kernel.org>
CC:     Michael Rodin <mrodin@...adit-jv.com>, <michael@...in.online>,
        <efriedrich@...adit-jv.com>, <erosca@...adit-jv.com>
Subject: [PATCH] media: v4l: vsp1: fix and generalize offset calculation for plane cropping

The vertical subsampling factor is currently not considered in the offset
calculation for plane cropping done in rpf_configure_partition. This causes
a distortion (shift of the color plane) when formats with the vsub factor
larger than 1 are used (e.g. NV12, see vsp1_video_formats in vsp1_pipe.c).
This commit considers vsub factor for all planes except plane 0
(luminance). Also this commit generalizes calculation of the offset, so
the formula is same for all of the planes.

Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
Signed-off-by: Michael Rodin <mrodin@...adit-jv.com>
---
 drivers/media/platform/vsp1/vsp1_rpf.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 85587c1..ea5503d 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -249,6 +249,7 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
 	const struct vsp1_format_info *fmtinfo = rpf->fmtinfo;
 	const struct v4l2_pix_format_mplane *format = &rpf->format;
 	struct v4l2_rect crop;
+	unsigned int i;
 
 	/*
 	 * Source size and crop offsets.
@@ -287,17 +288,13 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
 		       (crop.width << VI6_RPF_SRC_ESIZE_EHSIZE_SHIFT) |
 		       (crop.height << VI6_RPF_SRC_ESIZE_EVSIZE_SHIFT));
 
-	mem.addr[0] += crop.top * format->plane_fmt[0].bytesperline
-		     + crop.left * fmtinfo->bpp[0] / 8;
-
-	if (format->num_planes > 1) {
+	for (i = 0; i < format->num_planes; ++i) {
 		unsigned int offset;
 
-		offset = crop.top * format->plane_fmt[1].bytesperline
-		       + crop.left / fmtinfo->hsub
-		       * fmtinfo->bpp[1] / 8;
-		mem.addr[1] += offset;
-		mem.addr[2] += offset;
+		offset = crop.top * format->plane_fmt[i].bytesperline / (i ? fmtinfo->vsub : 1)
+		       + crop.left / (i ? fmtinfo->hsub : 1)
+		       * fmtinfo->bpp[i] / 8;
+		mem.addr[i] += offset;
 	}
 
 	/*
-- 
2.7.4