linux-kernel - [PATCH] drm/nouveau/kms: fix overflow in block size calculation in nouveau_check_bl

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20250507210550.141861-1-sdl@nppct.ru>
Date: Wed,  7 May 2025 21:05:47 +0000
From: Alexey Nepomnyashih <sdl@...ct.ru>
To: Lyude Paul <lyude@...hat.com>
Cc: Alexey Nepomnyashih <sdl@...ct.ru>,
	Danilo Krummrich <dakr@...nel.org>,
	David Airlie <airlied@...il.com>,
	Simona Vetter <simona@...ll.ch>,
	Ben Skeggs <bskeggs@...hat.com>,
	James Jones <jajones@...dia.com>,
	dri-devel@...ts.freedesktop.org,
	nouveau@...ts.freedesktop.org,
	linux-kernel@...r.kernel.org,
	lvc-project@...uxtesting.org
Subject: [PATCH] drm/nouveau/kms: fix overflow in block size calculation in nouveau_check_bl_size()

Prevent potential overflow in nouveau_check_bl_size() when calculating
bl_size. Although bl_size is a 64-bit value, the intermediate
multiplication of 32-bit operands (bw, bh, tile_mode, and gob_size) may
overflow before being assigned. gob_size is 256 or 512, and tile_mode is
validated to be ≤ 31, but bw and bh can still be large enough to trigger
overflow. Cast bw to uint64_t to ensure proper 64-bit arithmetic.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

Fixes: 4f5746c863db ("drm/nouveau/kms: Check framebuffer size against bo")
Signed-off-by: Alexey Nepomnyashih <sdl@...ct.ru>
---
 drivers/gpu/drm/nouveau/nouveau_display.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c
index add006fc8d81..0363711ee0ee 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.c
+++ b/drivers/gpu/drm/nouveau/nouveau_display.c
@@ -239,7 +239,7 @@ nouveau_check_bl_size(struct nouveau_drm *drm, struct nouveau_bo *nvbo,
 	bh = nouveau_get_height_in_blocks(h, gobs_in_block, drm->client.device.info.family);
 	gob_size = nouveau_get_gob_size(drm->client.device.info.family);
 
-	bl_size = bw * bh * gobs_in_block * gob_size;
+	bl_size = (uint64_t)bw * bh * gobs_in_block * gob_size;
 
 	DRM_DEBUG_KMS("offset=%u stride=%u h=%u gobs_in_block=%u bw=%u bh=%u gob_size=%u bl_size=%llu size=%zu\n",
 		      offset, stride, h, gobs_in_block, bw, bh, gob_size,
-- 
2.43.0