linux-kernel - [PATCH 2/2] drm/fb-helper: Do not scale depth down to 0

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 22 Mar 2019 15:56:13 -0500
From:   Reza Arbab <arbab@...ux.ibm.com>
To:     Dave Airlie <airlied@...hat.com>, Sean Paul <sean@...rly.run>,
        David Airlie <airlied@...ux.ie>,
        Daniel Vetter <daniel@...ll.ch>,
        Robert Tarasov <tutankhamen@...omium.org>,
        Linus Walleij <linus.walleij@...aro.org>,
        dri-devel@...ts.freedesktop.org, linux-kernel@...r.kernel.org
Cc:     wzabolot@...ktron.elka.pw.edu.pl, Pavel Machek <pavel@....cz>,
        Alex Deucher <alexander.deucher@....com>,
        Noralf Trønnes <noralf@...nnes.org>,
        Ville Syrjälä 
        <ville.syrjala@...ux.intel.com>
Subject: [PATCH 2/2] drm/fb-helper: Do not scale depth down to 0

If the scaling loop in drm_fb_helper_single_fb_probe() only finds
supported depths greater than what it's looking for, it will set
sizes.surface_depth = 0. This broke my udl device:

[drm] test CRTC 0 primary plane
udl 3-9.1:1.0: drm_fb_helper_single_fb_probe: test CRTC 0 format 0, depth=24
udl 3-9.1:1.0: drm_fb_helper_single_fb_probe: 24 > 16, skipping
udl 3-9.1:1.0: drm_fb_helper_single_fb_probe: test CRTC 0 format 1, depth=32
udl 3-9.1:1.0: drm_fb_helper_single_fb_probe: 32 > 16, skipping
[drm] requested bpp 16, scaled depth down to 0

Instead of setting zero depth, print that scaling was not possible and
do nothing.

Fixes: f4bd542bcaee ("drm/fb-helper: Scale back depth to supported maximum")
Signed-off-by: Reza Arbab <arbab@...ux.ibm.com>
---
 drivers/gpu/drm/drm_fb_helper.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 0e9349ff2d16..a7028655a065 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -1963,7 +1963,11 @@ static int drm_fb_helper_single_fb_probe(struct drm_fb_helper *fb_helper,
 				best_depth = fmt->depth;
 		}
 	}
-	if (sizes.surface_depth != best_depth) {
+
+	if (!best_depth) {
+		DRM_INFO("Could not scale depth to requested bpp %d",
+			 sizes.surface_bpp);
+	} else if (sizes.surface_depth != best_depth) {
 		DRM_INFO("requested bpp %d, scaled depth down to %d",
 			 sizes.surface_bpp, best_depth);
 		sizes.surface_depth = best_depth;
-- 
1.8.3.1