linux-kernel - [PATCH] drm/qxl: unbind vgacon

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20190220143640.9970-1-kraxel@redhat.com>
Date:   Wed, 20 Feb 2019 15:36:40 +0100
From:   Gerd Hoffmann <kraxel@...hat.com>
To:     dri-devel@...ts.freedesktop.org
Cc:     Gerd Hoffmann <kraxel@...hat.com>,
        Dave Airlie <airlied@...hat.com>,
        David Airlie <airlied@...ux.ie>,
        Daniel Vetter <daniel@...ll.ch>,
        virtualization@...ts.linux-foundation.org (open list:DRM DRIVER FOR QXL
        VIRTUAL GPU),
        spice-devel@...ts.freedesktop.org (open list:DRM DRIVER FOR QXL VIRTUAL
        GPU), linux-kernel@...r.kernel.org (open list)
Subject: [PATCH] drm/qxl: unbind vgacon

Problem:  qxl switches from native mode back into vga compatibility mode
when it notices someone is accessing vga registers.  And vgacon does
exactly that before fbcon takes over.

Before qxl switched to the generic fbdev emulation that didn't cause any
problems.  With the generic fbdev emulation the switch to vga mode
happens now and then, probably caused by a initialization order change
and triggered by a printk in a bad moment.

So make sure we take vgacon out of the picture by making dummycon
taking over the console early enough.

Not entriely happy with the approach, I'm open to better ideas.

Signed-off-by: Gerd Hoffmann <kraxel@...hat.com>
---
 drivers/gpu/drm/qxl/qxl_drv.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/gpu/drm/qxl/qxl_drv.c b/drivers/gpu/drm/qxl/qxl_drv.c
index bb81e310eb..88349dc13e 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.c
+++ b/drivers/gpu/drm/qxl/qxl_drv.c
@@ -30,6 +30,7 @@
 
 #include <linux/module.h>
 #include <linux/console.h>
+#include <linux/vt_kern.h>
 
 #include <drm/drmP.h>
 #include <drm/drm.h>
@@ -89,6 +90,11 @@ qxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	drm_kms_helper_poll_init(&qdev->ddev);
 
+	/* unbind vgacon to make sure it doesn't touch our vga registers */
+	console_lock();
+	ret = do_take_over_console(&dummy_con, 0, MAX_NR_CONSOLES - 1, true);
+	console_unlock();
+
 	/* Complete initialization. */
 	ret = drm_dev_register(&qdev->ddev, ent->driver_data);
 	if (ret)
-- 
2.9.3