linux-kernel - Re: [PATCH] usb-serial: Fix usb serial console open/close regression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4B951B0E.3000801@windriver.com>
Date:	Mon, 08 Mar 2010 09:43:10 -0600
From:	Jason Wessel <jason.wessel@...driver.com>
To:	Alan Stern <stern@...land.harvard.edu>
CC:	Greg Kroah-Hartman <gregkh@...e.de>,
	Alan Cox <alan@...ux.intel.com>,
	Oliver Neukum <oliver@...kum.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	linux-usb@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] usb-serial: Fix usb serial console open/close regression

Alan Stern wrote:
> On Mon, 8 Mar 2010, Jason Wessel wrote:
>
>   
>> Commit e1108a63e10d344284011cccc06328b2cd3e5da3 ("usb_serial: Use the
>> shutdown() operation") breaks the ability to use a usb console
>> starting in 2.6.32.  This was observed when using
>> console=ttyUSB0,115200 as a boot argument with an FTDI device.  The
>> error is:
>>
>> ftdi_sio ttyUSB0: ftdi_submit_read_urb - failed submitting read urb, error -22
>>
>> The handling of the ASYNCB_INITIALIZED changed in 2.6.32 such that in
>> tty_port_shutdown() it always clears the flag if it is set.  As a work
>> around the usb console must reset this flag, in order for the hw to
>> stay open.
>>
>> CC: Greg Kroah-Hartman <gregkh@...e.de>
>> CC: Alan Cox <alan@...ux.intel.com>
>> CC: Alan Stern <stern@...land.harvard.edu>
>> CC: Oliver Neukum <oliver@...kum.org>
>> CC: Andrew Morton <akpm@...ux-foundation.org>
>> CC: linux-usb@...r.kernel.org
>> CC: linux-kernel@...r.kernel.org
>> Signed-off-by: Jason Wessel <jason.wessel@...driver.com>
>>
>> ---
>>  drivers/usb/serial/usb-serial.c |    7 +++++--
>>  1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> --- a/drivers/usb/serial/usb-serial.c
>> +++ b/drivers/usb/serial/usb-serial.c
>> @@ -287,10 +287,13 @@ static void serial_down(struct tty_port 
>>  	struct usb_serial_driver *drv = port->serial->type;
>>  	/*
>>  	 * The console is magical.  Do not hang up the console hardware
>> -	 * or there will be tears.
>> +	 * or there will be tears.  If this is a console the initialized
>> +	 * flag is reset.
>>  	 */
>> -	if (port->console)
>> +	if (port->console) {
>> +		set_bit(ASYNCB_INITIALIZED, &port->port.flags);
>>  		return;
>> +	}
>>  	if (drv->close)
>>  		drv->close(port);
>>  }
>>     
>
> This is a little unfortunate.  It would be better to prevent
> tty_port_shutdown() from clearing ASYNCB_INITIALIZED in the first
> place.  The problem is that the tty core doesn't know when the port is
> being used as a console.  There ought to be a way to tell it.
>   

I agree, but presently there is no way to do so.   Up until 2.6.33 the
ASYNCB_INITIALIZED was being used to track this, but now it is used a
bit differently.

We still also have the same sort of issue for the passing in the initial
baud.  I don't know if you want to go for a short term approach this
way, or try to implement something different right now.  When you do
consider something longer term, I would like it to incorporate the other
serial settings as well, such that if the console initializes them first
the get correctly inherited by the tty open().

Jason.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/