linux-kernel - [PATCH v3] media: venus: add support for selection rectangles

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20200218184220.139656-1-jkardatzke@google.com>
Date:   Tue, 18 Feb 2020 10:42:16 -0800
From:   Jeffrey Kardatzke <jkardatzke@...gle.com>
To:     linux-media@...r.kernel.org
Cc:     Stanimir Varbanov <stanimir.varbanov@...aro.org>,
        Andy Gross <agross@...nel.org>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        linux-arm-msm@...r.kernel.org, linux-kernel@...r.kernel.org,
        Malathi Gottam <mgottam@...eaurora.org>,
        Jeffrey Kardatzke <jkardatzke@...gle.com>
Subject: [PATCH v3] media: venus: add support for selection rectangles

From: Malathi Gottam <>

Handles target type crop by setting the new active rectangle
to hardware. The new rectangle should be within YUV size.

This was taken from: https://lkml.org/lkml/2018/11/9/899

Signed-off-by: Malathi Gottam <mgottam@...eaurora.org>
Signed-off-by: Jeffrey Kardatzke <jkardatzke@...gle.com>
---
 drivers/media/platform/qcom/venus/venc.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 453edf966d4f..73b3181eed9a 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -479,10 +479,26 @@ venc_s_selection(struct file *file, void *fh, struct v4l2_selection *s)
 
 	switch (s->target) {
 	case V4L2_SEL_TGT_CROP:
-		if (s->r.width != inst->out_width ||
-		    s->r.height != inst->out_height ||
-		    s->r.top != 0 || s->r.left != 0)
-			return -EINVAL;
+		if (s->r.left != 0) {
+			s->r.width += s->r.left;
+			s->r.left = 0;
+		}
+
+		if (s->r.top != 0) {
+			s->r.height += s->r.top;
+			s->r.top = 0;
+		}
+
+		if (s->r.width > inst->width)
+			s->r.width = inst->width;
+		else
+			inst->width = s->r.width;
+
+		if (s->r.height > inst->height)
+			s->r.height = inst->height;
+		else
+			inst->height = s->r.height;
+
 		break;
 	default:
 		return -EINVAL;
-- 
2.25.0.265.gbab2e86ba0-goog