linux-kernel - [PATCH v6 06/21] drm/display: hdmi-state-helper: Try subsampling in mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20260121-color-format-v6-6-7b81a771cd0b@collabora.com>
Date: Wed, 21 Jan 2026 14:45:32 +0100
From: Nicolas Frattaroli <nicolas.frattaroli@...labora.com>
To: Harry Wentland <harry.wentland@....com>, Leo Li <sunpeng.li@....com>, 
 Rodrigo Siqueira <siqueira@...lia.com>, 
 Alex Deucher <alexander.deucher@....com>, 
 Christian König <christian.koenig@....com>, 
 David Airlie <airlied@...il.com>, Simona Vetter <simona@...ll.ch>, 
 Maarten Lankhorst <maarten.lankhorst@...ux.intel.com>, 
 Maxime Ripard <mripard@...nel.org>, Thomas Zimmermann <tzimmermann@...e.de>, 
 Andrzej Hajda <andrzej.hajda@...el.com>, 
 Neil Armstrong <neil.armstrong@...aro.org>, Robert Foss <rfoss@...nel.org>, 
 Laurent Pinchart <Laurent.pinchart@...asonboard.com>, 
 Jonas Karlman <jonas@...boo.se>, Jernej Skrabec <jernej.skrabec@...il.com>, 
 Sandy Huang <hjc@...k-chips.com>, 
 Heiko Stübner <heiko@...ech.de>, 
 Andy Yan <andy.yan@...k-chips.com>, 
 Jani Nikula <jani.nikula@...ux.intel.com>, 
 Rodrigo Vivi <rodrigo.vivi@...el.com>, 
 Joonas Lahtinen <joonas.lahtinen@...ux.intel.com>, 
 Tvrtko Ursulin <tursulin@...ulin.net>, Dmitry Baryshkov <lumag@...nel.org>, 
 Sascha Hauer <s.hauer@...gutronix.de>, Rob Herring <robh@...nel.org>, 
 Jonathan Corbet <corbet@....net>
Cc: kernel@...labora.com, amd-gfx@...ts.freedesktop.org, 
 dri-devel@...ts.freedesktop.org, linux-kernel@...r.kernel.org, 
 linux-arm-kernel@...ts.infradead.org, linux-rockchip@...ts.infradead.org, 
 intel-gfx@...ts.freedesktop.org, intel-xe@...ts.freedesktop.org, 
 linux-doc@...r.kernel.org, 
 Nicolas Frattaroli <nicolas.frattaroli@...labora.com>
Subject: [PATCH v6 06/21] drm/display: hdmi-state-helper: Try subsampling
 in mode_valid

drm_hdmi_connector_mode_valid assumes modes are only valid if they work
with RGB. The reality is more complex however: YCbCr 4:2:0
chroma-subsampled modes only require half the pixel clock that the same
mode would require in RGB.

This leads to drm_hdmi_connector_mode_valid rejecting perfectly valid
420-only or 420-also modes.

Fix this by checking whether the mode is 420-capable first. If so, then
proceed by checking it with HDMI_COLORSPACE_YUV420 so long as the
connector has legalized 420, otherwise error out. If the mode is not
420-capable, check with RGB as was previously always the case.

Fixes: 47368ab437fd ("drm/display: hdmi: add generic mode_valid helper")
Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@...labora.com>
---
 drivers/gpu/drm/display/drm_hdmi_state_helper.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/display/drm_hdmi_state_helper.c b/drivers/gpu/drm/display/drm_hdmi_state_helper.c
index 1ea3b9c93aa5..ceabfdb30f7a 100644
--- a/drivers/gpu/drm/display/drm_hdmi_state_helper.c
+++ b/drivers/gpu/drm/display/drm_hdmi_state_helper.c
@@ -898,8 +898,21 @@ drm_hdmi_connector_mode_valid(struct drm_connector *connector,
 			      const struct drm_display_mode *mode)
 {
 	unsigned long long clock;
+	enum hdmi_colorspace fmt;
+
+	if (drm_mode_is_420_only(&connector->display_info, mode)) {
+		if (connector->ycbcr_420_allowed)
+			fmt = HDMI_COLORSPACE_YUV420;
+		else
+			return MODE_NO_420;
+	} else if (drm_mode_is_420_also(&connector->display_info, mode) &&
+		   connector->ycbcr_420_allowed) {
+		fmt = HDMI_COLORSPACE_YUV420;
+	} else {
+		fmt = HDMI_COLORSPACE_RGB;
+	}
 
-	clock = drm_hdmi_compute_mode_clock(mode, 8, HDMI_COLORSPACE_RGB);
+	clock = drm_hdmi_compute_mode_clock(mode, 8, fmt);
 	if (!clock)
 		return MODE_ERROR;
 

-- 
2.52.0