linux-kernel - [PATCH 1/3] usb,console: Fix regression in usb console on kernel boot

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1245291609-14871-2-git-send-email-jason.wessel@windriver.com>
Date:	Wed, 17 Jun 2009 21:20:07 -0500
From:	Jason Wessel <jason.wessel@...driver.com>
To:	greg@...ah.com, alan@...rguk.ukuu.org.uk
Cc:	linux-kernel@...r.kernel.org,
	Jason Wessel <jason.wessel@...driver.com>
Subject: [PATCH 1/3] usb,console: Fix regression in usb console on kernel boot

The commit 335f8514f200e63d689113d29cb7253a5c282967 introduced a
regression which stopped usb consoles from working correctly as a
kernel boot console as well as interactive login device.

The addition of the serial_close() which in turn calls
tty_port_close_start() will change the reference count of port.count
and warn about it.  The usb console code had previously incremented
the port.count to indicate it was making use of the device as a
console and the forced change causes a double open on the usb device
which leads to a non obvious kernel oops later on when the tty is
freed.

To fix the problem instead make use of port->console to track if the
port is in fact an active console port to avoid double initialization
of the usb serial device.  The port.count is incremented and
decremented only with in the scope of usb_console_setup() for the
purpose of the low level driver initialization.

Signed-off-by: Jason Wessel <jason.wessel@...driver.com>
---
 drivers/usb/serial/console.c    |   13 +++++++------
 drivers/usb/serial/usb-serial.c |    3 ++-
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/usb/serial/console.c b/drivers/usb/serial/console.c
index 247b61b..0e4f2e4 100644
--- a/drivers/usb/serial/console.c
+++ b/drivers/usb/serial/console.c
@@ -169,9 +169,11 @@ static int usb_console_setup(struct console *co, char *options)
 			kfree(tty);
 		}
 	}
-	/* So we know not to kill the hardware on a hangup on this
-	   port. We have also bumped the use count by one so it won't go
-	   idle */
+	/* Now that any required fake tty operations are completed restore
+	 * the tty port count */
+	--port->port.count;
+	/* The console is special in terms of closing the device so
+	 * indicate this port is now acting as a system console. */
 	port->console = 1;
 	retval = 0;
 
@@ -204,7 +206,7 @@ static void usb_console_write(struct console *co,
 
 	dbg("%s - port %d, %d byte(s)", __func__, port->number, count);
 
-	if (!port->port.count) {
+	if (!port->console) {
 		dbg("%s - port not opened", __func__);
 		return;
 	}
@@ -300,8 +302,7 @@ void usb_serial_console_exit(void)
 {
 	if (usbcons_info.port) {
 		unregister_console(&usbcons);
-		if (usbcons_info.port->port.count)
-			usbcons_info.port->port.count--;
+		usbcons_info.port->console = 0;
 		usbcons_info.port = NULL;
 	}
 }
diff --git a/drivers/usb/serial/usb-serial.c b/drivers/usb/serial/usb-serial.c
index d595aa5..329e098 100644
--- a/drivers/usb/serial/usb-serial.c
+++ b/drivers/usb/serial/usb-serial.c
@@ -220,7 +220,8 @@ static int serial_open (struct tty_struct *tty, struct file *filp)
 	tty->driver_data = port;
 	tty_port_tty_set(&port->port, tty);
 
-	if (port->port.count == 1) {
+	/* If the console is attached, the device is already open */
+	if (port->port.count == 1 && !port->console) {
 
 		/* lock this module before we call it
 		 * this may fail, which means we must bail out,
-- 
1.6.3.1.9.g95405b

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/