linux-kernel - [PATCH 1/2] tty_port,usb-console: Fix usb serial console open/close regression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1268106612-6854-1-git-send-email-jason.wessel@windriver.com>
Date:	Mon,  8 Mar 2010 21:50:11 -0600
From:	Jason Wessel <jason.wessel@...driver.com>
To:	Greg Kroah-Hartman <gregkh@...e.de>
Cc:	Jason Wessel <jason.wessel@...driver.com>,
	Greg Kroah-Hartman <gregkh@...e.de>,
	Alan Cox <alan@...ux.intel.com>,
	Alan Stern <stern@...land.harvard.edu>,
	Oliver Neukum <oliver@...kum.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	linux-usb@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: [PATCH 1/2] tty_port,usb-console: Fix usb serial console open/close regression

Commit e1108a63e10d344284011cccc06328b2cd3e5da3 ("usb_serial: Use the
shutdown() operation") breaks the ability to use a usb console
starting in 2.6.33.  This was observed when using
console=ttyUSB0,115200 as a boot argument with an FTDI device.  The
error is:

ftdi_sio ttyUSB0: ftdi_submit_read_urb - failed submitting read urb, error -22

The handling of the ASYNCB_INITIALIZED changed in 2.6.32 such that in
tty_port_shutdown() it always clears the flag if it is set.  The fix
is to add a variable to the tty_port struct to indicate when the tty
port is a console.

CC: Greg Kroah-Hartman <gregkh@...e.de>
CC: Alan Cox <alan@...ux.intel.com>
CC: Alan Stern <stern@...land.harvard.edu>
CC: Oliver Neukum <oliver@...kum.org>
CC: Andrew Morton <akpm@...ux-foundation.org>
CC: linux-usb@...r.kernel.org
CC: linux-kernel@...r.kernel.org
Signed-off-by: Jason Wessel <jason.wessel@...driver.com>
---
 drivers/char/tty_port.c      |    2 +-
 drivers/usb/serial/console.c |    1 +
 include/linux/tty.h          |    1 +
 3 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/drivers/char/tty_port.c b/drivers/char/tty_port.c
index be492dd..a3bd1d0 100644
--- a/drivers/char/tty_port.c
+++ b/drivers/char/tty_port.c
@@ -119,7 +119,7 @@ EXPORT_SYMBOL(tty_port_tty_set);
 static void tty_port_shutdown(struct tty_port *port)
 {
 	mutex_lock(&port->mutex);
-	if (port->ops->shutdown &&
+	if (port->ops->shutdown && !port->console &&
 		test_and_clear_bit(ASYNCB_INITIALIZED, &port->flags))
 			port->ops->shutdown(port);
 	mutex_unlock(&port->mutex);
diff --git a/drivers/usb/serial/console.c b/drivers/usb/serial/console.c
index b22ac32..f347da2 100644
--- a/drivers/usb/serial/console.c
+++ b/drivers/usb/serial/console.c
@@ -181,6 +181,7 @@ static int usb_console_setup(struct console *co, char *options)
 	/* The console is special in terms of closing the device so
 	 * indicate this port is now acting as a system console. */
 	port->console = 1;
+	port->port.console = 1;
 
 	mutex_unlock(&serial->disc_mutex);
 	return retval;
diff --git a/include/linux/tty.h b/include/linux/tty.h
index d96e588..a42d7a6 100644
--- a/include/linux/tty.h
+++ b/include/linux/tty.h
@@ -223,6 +223,7 @@ struct tty_port {
 	wait_queue_head_t	close_wait;	/* Close waiters */
 	wait_queue_head_t	delta_msr_wait;	/* Modem status change */
 	unsigned long		flags;		/* TTY flags ASY_*/
+	unsigned char		console:1;	/* port is a console */
 	struct mutex		mutex;		/* Locking */
 	struct mutex		buf_mutex;	/* Buffer alloc lock */
 	unsigned char		*xmit_buf;	/* Optional buffer */
-- 
1.5.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/