linux-kernel - [PATCH v2] media: v4l2-ctrls: add full AV1 profile validation in validate_av1

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20250913103939.25658-1-opensource206@gmail.com>
Date: Sat, 13 Sep 2025 16:09:27 +0530
From: Pavan Bobba <opensource206@...il.com>
To: mchehab@...nel.org,
	hverkuil@...nel.org,
	ribalda@...omium.org,
	laurent.pinchart@...asonboard.com,
	yunkec@...gle.com,
	sakari.ailus@...ux.intel.com,
	james.cowgill@...ize.com
Cc: linux-media@...r.kernel.org,
	linux-kernel@...r.kernel.org,
	Pavan Bobba <opensource206@...il.com>
Subject: [PATCH v2] media: v4l2-ctrls: add full AV1 profile validation in validate_av1_sequence()

Complete the "TODO: PROFILES" by enforcing profile-specific and
monochrome constraints as defined by the AV1 specification
(Section 5.5.2, "Color config syntax").

The validator now checks:

 - Flags: reject any unknown bits set in sequence->flags
 - Profile range: only profiles 0..2 are valid
 - Profile 0: 8/10-bit only, subsampling must be 4:2:0 (sx=1, sy=1),
   monochrome allowed
 - Profile 1: 8/10-bit only, subsampling must be 4:4:4 (sx=0, sy=0),
   monochrome forbidden
 - Profile 2:
    * 8/10-bit: only 4:2:2 allowed (sx=1, sy=0)
    * 12-bit: 4:4:4 (sx=0, sy=0), 4:2:2 (sx=1, sy=0), or 4:2:0 (sx=1, sy=1)
      allowed
 - Monochrome path (all profiles except 1): forces subsampling_x=1,
   subsampling_y=1, separate_uv_delta_q=0

These checks prevent userspace from providing invalid AV1 sequence
headers that would otherwise be accepted, leading to undefined driver
or hardware behavior.

Signed-off-by: Pavan Bobba <opensource206@...il.com>
---
v1 -> v2 : Added more checks for subsampling combinations per profile.
          : Added a TODO note in the function header for checks to be implemented later.

 drivers/media/v4l2-core/v4l2-ctrls-core.c | 54 +++++++++++++++++++++--
 1 file changed, 50 insertions(+), 4 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 98b960775e87..3283ed04cc36 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -852,14 +852,60 @@ static int validate_av1_sequence(struct v4l2_ctrl_av1_sequence *s)
 	 V4L2_AV1_SEQUENCE_FLAG_SEPARATE_UV_DELTA_Q))
 		return -EINVAL;
 
-	if (s->seq_profile == 1 && s->flags & V4L2_AV1_SEQUENCE_FLAG_MONO_CHROME)
-		return -EINVAL;
-
 	/* reserved */
 	if (s->seq_profile > 2)
 		return -EINVAL;
 
-	/* TODO: PROFILES */
+	/* Profile-specific checks */
+	switch (s->seq_profile) {
+	case 0:
+		/* Bit depth: 8 or 10 */
+		if (s->bit_depth != 8 && s->bit_depth != 10)
+			return -EINVAL;
+
+		/* Subsampling must be 4:2:0 → x=1, y=1 */
+		if (!(s->flags & V4L2_AV1_SEQUENCE_FLAG_SUBSAMPLING_X) ||
+		    !(s->flags & V4L2_AV1_SEQUENCE_FLAG_SUBSAMPLING_Y))
+			return -EINVAL;
+		break;
+
+	case 1:
+		/* Monochrome is forbidden in profile 1 */
+		if (s->flags & V4L2_AV1_SEQUENCE_FLAG_MONO_CHROME)
+			return -EINVAL;
+
+		/* Bit depth: 8 or 10 */
+		if (s->bit_depth != 8 && s->bit_depth != 10)
+			return -EINVAL;
+
+		/* Subsampling must be 4:4:4 → x=0, y=0 */
+		if ((s->flags & V4L2_AV1_SEQUENCE_FLAG_SUBSAMPLING_X) ||
+		    (s->flags & V4L2_AV1_SEQUENCE_FLAG_SUBSAMPLING_Y))
+			return -EINVAL;
+		break;
+
+	case 2:
+		/* Bit depth: 8, 10, or 12 */
+		if (s->bit_depth != 8 && s->bit_depth != 10 &&
+		    s->bit_depth != 12)
+			return -EINVAL;
+
+		/* Subsampling: 4:2:0, 4:2:2, or 4:4:4 allowed → no extra check */
+		break;
+	}
+
+	/* If monochrome flag is set, enforce spec rules */
+	if (s->flags & V4L2_AV1_SEQUENCE_FLAG_MONO_CHROME) {
+		/* Must signal subsampling_x=1, subsampling_y=1 */
+		if (!(s->flags & V4L2_AV1_SEQUENCE_FLAG_SUBSAMPLING_X) ||
+		    !(s->flags & V4L2_AV1_SEQUENCE_FLAG_SUBSAMPLING_Y))
+			return -EINVAL;
+
+		/* separate_uv_delta_q must be 0 in monochrome */
+		if (s->flags & V4L2_AV1_SEQUENCE_FLAG_SEPARATE_UV_DELTA_Q)
+			return -EINVAL;
+	}
+
 	return 0;
 }
 
-- 
2.43.0