linux-kernel - [PATCH 4.14 066/140] drm/nouveau: prefer XBGR2101010 for addfb ioctl

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20180313152502.645542034@linuxfoundation.org>
Date:   Tue, 13 Mar 2018 16:24:29 +0100
From:   Greg Kroah-Hartman <gregkh@...uxfoundation.org>
To:     linux-kernel@...r.kernel.org
Cc:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        stable@...r.kernel.org, Ilia Mirkin <imirkin@...m.mit.edu>,
        Ben Skeggs <bskeggs@...hat.com>,
        Daniel Vetter <daniel.vetter@...ll.ch>
Subject: [PATCH 4.14 066/140] drm/nouveau: prefer XBGR2101010 for addfb ioctl

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Ilia Mirkin <imirkin@...m.mit.edu>

commit c20bb155c2c5acb775f68be5d84fe679687c3c1e upstream.

Nouveau only exposes support for XBGR2101010. Prior to the atomic
conversion, drm would pass in the wrong format in the framebuffer, but
it was always ignored -- both userspace (xf86-video-nouveau) and the
kernel driver agreed on the layout, so the fact that the format was
wrong didn't matter.

With the atomic conversion, nouveau all of a sudden started caring about
the exact format, and so the previously-working code in
xf86-video-nouveau no longer functioned since the (internally-assigned)
format from the addfb ioctl was wrong.

This change adds infrastructure to allow a drm driver to specify that it
prefers the XBGR format variant for the addfb ioctl, and makes nouveau's
nv50 display driver set it. (Prior gens had no support for 30bpp at all.)

Signed-off-by: Ilia Mirkin <imirkin@...m.mit.edu>
Cc: stable@...r.kernel.org # v4.10+
Acked-by: Ben Skeggs <bskeggs@...hat.com>
Signed-off-by: Daniel Vetter <daniel.vetter@...ll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20180203191123.31507-1-imirkin@alum.mit.edu
Signed-off-by: Greg Kroah-Hartman <gregkh@...uxfoundation.org>

---
 drivers/gpu/drm/drm_framebuffer.c      |    4 ++++
 drivers/gpu/drm/nouveau/nv50_display.c |    1 +
 include/drm/drm_drv.h                  |    1 +
 3 files changed, 6 insertions(+)

--- a/drivers/gpu/drm/drm_framebuffer.c
+++ b/drivers/gpu/drm/drm_framebuffer.c
@@ -118,6 +118,10 @@ int drm_mode_addfb(struct drm_device *de
 	r.pixel_format = drm_mode_legacy_fb_format(or->bpp, or->depth);
 	r.handles[0] = or->handle;
 
+	if (r.pixel_format == DRM_FORMAT_XRGB2101010 &&
+	    dev->driver->driver_features & DRIVER_PREFER_XBGR_30BPP)
+		r.pixel_format = DRM_FORMAT_XBGR2101010;
+
 	ret = drm_mode_addfb2(dev, &r, file_priv);
 	if (ret)
 		return ret;
--- a/drivers/gpu/drm/nouveau/nv50_display.c
+++ b/drivers/gpu/drm/nouveau/nv50_display.c
@@ -4426,6 +4426,7 @@ nv50_display_create(struct drm_device *d
 	nouveau_display(dev)->fini = nv50_display_fini;
 	disp->disp = &nouveau_display(dev)->disp;
 	dev->mode_config.funcs = &nv50_disp_func;
+	dev->driver->driver_features |= DRIVER_PREFER_XBGR_30BPP;
 	if (nouveau_atomic)
 		dev->driver->driver_features |= DRIVER_ATOMIC;
 
--- a/include/drm/drm_drv.h
+++ b/include/drm/drm_drv.h
@@ -55,6 +55,7 @@ struct drm_mode_create_dumb;
 #define DRIVER_ATOMIC			0x10000
 #define DRIVER_KMS_LEGACY_CONTEXT	0x20000
 #define DRIVER_SYNCOBJ                  0x40000
+#define DRIVER_PREFER_XBGR_30BPP        0x80000
 
 /**
  * struct drm_driver - DRM driver structure