linux-kernel - Re: [PATCH v5 06/17] drm/display: hdmi-state-helper: Try subsampling in mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20251209-dramatic-caiman-of-luck-db9d0f@houat>
Date: Tue, 9 Dec 2025 15:18:25 +0100
From: Maxime Ripard <mripard@...nel.org>
To: Nicolas Frattaroli <nicolas.frattaroli@...labora.com>
Cc: Harry Wentland <harry.wentland@....com>, Leo Li <sunpeng.li@....com>, 
	Rodrigo Siqueira <siqueira@...lia.com>, Alex Deucher <alexander.deucher@....com>, 
	Christian König <christian.koenig@....com>, David Airlie <airlied@...il.com>, 
	Simona Vetter <simona@...ll.ch>, Maarten Lankhorst <maarten.lankhorst@...ux.intel.com>, 
	Thomas Zimmermann <tzimmermann@...e.de>, Andrzej Hajda <andrzej.hajda@...el.com>, 
	Neil Armstrong <neil.armstrong@...aro.org>, Robert Foss <rfoss@...nel.org>, 
	Laurent Pinchart <Laurent.pinchart@...asonboard.com>, Jonas Karlman <jonas@...boo.se>, 
	Jernej Skrabec <jernej.skrabec@...il.com>, Sandy Huang <hjc@...k-chips.com>, 
	Heiko Stübner <heiko@...ech.de>, Andy Yan <andy.yan@...k-chips.com>, 
	Jani Nikula <jani.nikula@...ux.intel.com>, Rodrigo Vivi <rodrigo.vivi@...el.com>, 
	Joonas Lahtinen <joonas.lahtinen@...ux.intel.com>, Tvrtko Ursulin <tursulin@...ulin.net>, 
	Dmitry Baryshkov <lumag@...nel.org>, Sascha Hauer <s.hauer@...gutronix.de>, 
	Rob Herring <robh@...nel.org>, kernel@...labora.com, amd-gfx@...ts.freedesktop.org, 
	dri-devel@...ts.freedesktop.org, linux-kernel@...r.kernel.org, 
	linux-arm-kernel@...ts.infradead.org, linux-rockchip@...ts.infradead.org, 
	intel-gfx@...ts.freedesktop.org, intel-xe@...ts.freedesktop.org
Subject: Re: [PATCH v5 06/17] drm/display: hdmi-state-helper: Try subsampling
 in mode_valid

On Fri, Nov 28, 2025 at 10:05:42PM +0100, Nicolas Frattaroli wrote:
> drm_hdmi_connector_mode_valid assumes modes are only valid if they work
> with RGB. The reality is more complex however: YCbCr 4:2:0
> chroma-subsampled modes only require half the pixel clock that the same
> mode would require in RGB.
> 
> This leads to drm_hdmi_connector_mode_valid rejecting perfectly valid
> 420-only modes.
> 
> Fix this by checking whether the mode is 420-only first. If so, then
> proceed by checking it with HDMI_COLORSPACE_YUV420 so long as the
> connector has legalized 420, otherwise error out. If the mode is not
> 420-only, check with RGB as was previously always the case.
> 
> Fixes: 47368ab437fd ("drm/display: hdmi: add generic mode_valid helper")
> Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@...labora.com>
> ---
>  drivers/gpu/drm/display/drm_hdmi_state_helper.c | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/display/drm_hdmi_state_helper.c b/drivers/gpu/drm/display/drm_hdmi_state_helper.c
> index 5da956bdd68c..1800e00b30c5 100644
> --- a/drivers/gpu/drm/display/drm_hdmi_state_helper.c
> +++ b/drivers/gpu/drm/display/drm_hdmi_state_helper.c
> @@ -892,8 +892,18 @@ drm_hdmi_connector_mode_valid(struct drm_connector *connector,
>  			      const struct drm_display_mode *mode)
>  {
>  	unsigned long long clock;
> +	enum hdmi_colorspace fmt;
> +
> +	if (drm_mode_is_420_only(&connector->display_info, mode)) {
> +		if (connector->ycbcr_420_allowed)
> +			fmt = HDMI_COLORSPACE_YUV420;
> +		else
> +			return MODE_NO_420;
> +	} else {
> +		fmt = HDMI_COLORSPACE_RGB;
> +	}
>  
> -	clock = drm_hdmi_compute_mode_clock(mode, 8, HDMI_COLORSPACE_RGB);
> +	clock = drm_hdmi_compute_mode_clock(mode, 8, fmt);

I agree on principle, but we need to have a test for this.

Maxime

Download attachment "signature.asc" of type "application/pgp-signature" (274 bytes)