linux-kernel - Re: [PATCH] drm/msm: fix allocation of dumb buffers for non-RGB formats

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <5aa617b1-f6b8-49a3-9659-1b4d35c89d22@suse.de>
Date: Mon, 3 Nov 2025 09:09:09 +0100
From: Thomas Zimmermann <tzimmermann@...e.de>
To: Dmitry Baryshkov <dmitry.baryshkov@....qualcomm.com>,
 Rob Clark <robin.clark@....qualcomm.com>, Dmitry Baryshkov
 <lumag@...nel.org>, Abhinav Kumar <abhinav.kumar@...ux.dev>,
 Jessica Zhang <jesszhan0024@...il.com>, Sean Paul <sean@...rly.run>,
 Marijn Suijten <marijn.suijten@...ainline.org>,
 David Airlie <airlied@...il.com>, Simona Vetter <simona@...ll.ch>
Cc: linux-arm-msm@...r.kernel.org, dri-devel@...ts.freedesktop.org,
 freedreno@...ts.freedesktop.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] drm/msm: fix allocation of dumb buffers for non-RGB
 formats

Hi,

thanks a lot for this bug fix.

Am 02.11.25 um 21:17 schrieb Dmitry Baryshkov:
> Several users (including IGT kms_getfb tests) allocate DUMB buffers for
> YUV data. Commit 538fa012cbdb ("drm/msm: Compute dumb-buffer sizes with
> drm_mode_size_dumb()") broke that usecase, since in those cases
> drm_driver_color_mode_format() returns DRM_FORMAT_INVALID.
>
> Handle the YUV usecase, aligning to 32-bit pixels.
>
> Fixes: 538fa012cbdb ("drm/msm: Compute dumb-buffer sizes with drm_mode_size_dumb()")
> Closes: https://lore.kernel.org/all/vptw5tquup34e3jen62znnw26qe76f3pys4lpsal5g3czwev6y@2q724ibos7by/
> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@....qualcomm.com>
> ---
>   drivers/gpu/drm/msm/msm_gem.c | 17 ++++++++++-------
>   1 file changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
> index 435c0067c2641786ca7b4a42cca0ae7e7dda3451..b9cfdb2ff33b63b92cff2568db491d0692bbe459 100644
> --- a/drivers/gpu/drm/msm/msm_gem.c
> +++ b/drivers/gpu/drm/msm/msm_gem.c
> @@ -701,7 +701,6 @@ int msm_gem_dumb_create(struct drm_file *file, struct drm_device *dev,
>   		struct drm_mode_create_dumb *args)
>   {
>   	u32 fourcc;
> -	const struct drm_format_info *info;
>   	u64 pitch_align;
>   	int ret;
>   
> @@ -711,12 +710,16 @@ int msm_gem_dumb_create(struct drm_file *file, struct drm_device *dev,
>   	 * Use the result as pitch alignment.
>   	 */
>   	fourcc = drm_driver_color_mode_format(dev, args->bpp);
> -	if (fourcc == DRM_FORMAT_INVALID)
> -		return -EINVAL;
> -	info = drm_format_info(fourcc);
> -	if (!info)
> -		return -EINVAL;
> -	pitch_align = drm_format_info_min_pitch(info, 0, SZ_32);
> +	if (fourcc != DRM_FORMAT_INVALID) {
> +		const struct drm_format_info *info;
> +
> +		info = drm_format_info(fourcc);
> +		if (!info)
> +			return -EINVAL;
> +		pitch_align = drm_format_info_min_pitch(info, 0, SZ_32);
> +	} else {
> +		pitch_align = round_up(args->width, SZ_32) * DIV_ROUND_UP(args->bpp, SZ_8);

Just a style issue: I know that the fixed patch uses SZ_32, but looking 
at now, it seems preferable to use a plain 32 here. It's not a size 
value in the common sense.

The same problem happens in imx (see patch 10 of the dumb-buffer 
series). I think we could use a helper for the code. Something like

u64 drm_mode_dumb_pitch_align(const struct drm_mode_create_dumb *args, 
unsigned int pitch_pixel_align)
{
     // do the fixed calculation here

     // return 0 on error
}


But if you just want to get the bug fixed, feel free to add

Reviewed-by: Thomas Zimmermann <tzimmermann@...e.de>

to the current patch.

Best regards
Thomas

> +	}
>   	if (!pitch_align || pitch_align > U32_MAX)
>   		return -EINVAL;
>   	ret = drm_mode_size_dumb(dev, args, pitch_align, 0);
>
> ---
> base-commit: 98bd8b16ae57e8f25c95d496fcde3dfdd8223d41
> change-id: 20251102-drm-msm-fix-nv12-11f0e8c31b2e
>
> Best regards,

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)