linux-kernel - [PATCH 4.6 138/203] drm/vmwgfx: Work around mode set failure in 2D VMs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 25 Jul 2016 13:55:53 -0700
From:	Greg Kroah-Hartman <gregkh@...uxfoundation.org>
To:	linux-kernel@...r.kernel.org
Cc:	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	stable@...r.kernel.org, Sinclair Yeh <syeh@...are.com>,
	Thomas Hellstrom <thellstrom@...are.com>
Subject: [PATCH 4.6 138/203] drm/vmwgfx: Work around mode set failure in 2D VMs

4.6-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Sinclair Yeh <syeh@...are.com>

commit 7c20d213dd3cd6295bf9162730e7a368af957854 upstream.

In a low-memory 2D VM, fbdev can take up a large percentage of
available memory, making them unavailable for other DRM clients.

Since we do not take fbdev into account when filtering modes,
we end up claiming to support more modes than we actually do.

As a result, users get a black screen when setting a mode too
large for current available memory.  In a low-memory VM
configuration, users can get a black screen for a mode as low
as 1024x768.

The current mode filtering mechanism keys off of
SVGA_REG_SUGGESTED_GBOBJECT_MEM_SIZE_KB, i.e. the maximum amount
of surface memory we have.  Since this value is a performance
suggestion, not a hard limit, and since there should not be much
of a performance impact for a 2D VM, rather than filtering out
more modes, we will just allow ourselves to exceed the SVGA's
performance suggestion.

Also changed assumed bpp to 32 from 16 to make sure we can
actually support all the modes listed.

Signed-off-by: Sinclair Yeh <syeh@...are.com>
Reviewed-by: Thomas Hellstrom <thellstrom@...are.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@...uxfoundation.org>

---
 drivers/gpu/drm/vmwgfx/vmwgfx_drv.c |    7 +++++++
 drivers/gpu/drm/vmwgfx/vmwgfx_kms.c |    9 +--------
 2 files changed, 8 insertions(+), 8 deletions(-)

--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
@@ -704,6 +704,13 @@ static int vmw_driver_load(struct drm_de
 			vmw_read(dev_priv,
 				 SVGA_REG_SUGGESTED_GBOBJECT_MEM_SIZE_KB);

+		/*
+		 * Workaround for low memory 2D VMs to compensate for the
+		 * allocation taken by fbdev
+		 */
+		if (!(dev_priv->capabilities & SVGA_CAP_3D))
+			mem_size *= 2;
+
 		dev_priv->max_mob_pages = mem_size * 1024 / PAGE_SIZE;
 		dev_priv->prim_bb_mem =
 			vmw_read(dev_priv,
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
@@ -1553,14 +1553,7 @@ int vmw_du_connector_fill_modes(struct d
 		DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_PVSYNC)
 	};
 	int i;
-	u32 assumed_bpp = 2;
-
-	/*
-	 * If using screen objects, then assume 32-bpp because that's what the
-	 * SVGA device is assuming
-	 */
-	if (dev_priv->active_display_unit == vmw_du_screen_object)
-		assumed_bpp = 4;
+	u32 assumed_bpp = 4;

 	if (dev_priv->assume_16bpp)
 		assumed_bpp = 2;