linux-kernel - Re: linux-next: Tree for March 12 (tty)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 14 Mar 2009 14:11:49 +0100
From:	Bartlomiej Zolnierkiewicz <bzolnier@...il.com>
To:	Stephen Rothwell <sfr@...b.auug.org.au>,
	Alan Cox <alan@...rguk.ukuu.org.uk>
Cc:	linux-next@...r.kernel.org, LKML <linux-kernel@...r.kernel.org>
Subject: Re: linux-next: Tree for March 12 (tty)


Hi,

On Thursday 12 March 2009, Stephen Rothwell wrote:
> Hi all,
> 
> Changes since 20090311:

It fails to boot here (both with QEMU and the real hardware):

...
Warning: unable to open an initial console.
Kernel panic - not syncing: Attempted to kill init!
...

I bisected the problem to:

>From 31f8326f981117fc548d292656fa8dbed28bc42b Mon Sep 17 00:00:00 2001
From: Alan Cox <alan@...rguk.ukuu.org.uk>
Date: Thu, 12 Mar 2009 10:00:15 +1100
Subject: [PATCH] tty-fix-mismatch-in-lookup-han

The lookup methods return NULL or a pointer. The caller expects an ERR_PTR.
It seems to make sense to just use NULL and return -ENODEV here as it keeps
the code simple and clean and the way it always used to work (effectively)
before the restructure

Problem noticed by Jiri Slaby.

Signed-off-by: Alan Cox <alan@...rguk.ukuu.org.uk>
---
 drivers/char/tty_io.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/char/tty_io.c b/drivers/char/tty_io.c
index bc84e12..0cc0c60 100644
--- a/drivers/char/tty_io.c
+++ b/drivers/char/tty_io.c
@@ -1817,9 +1817,9 @@ got_driver:
 		/* check whether we're reopening an existing tty */
 		tty = tty_driver_lookup_tty(driver, inode, index);
 
-		if (IS_ERR(tty)) {
+		if (tty == NULL) {
 			mutex_unlock(&tty_mutex);
-			return PTR_ERR(tty);
+			return -ENODEV;
 		}
 	}
 
-- 

It seems that fixing IS_ERR() check uncovered some underlying problem
since we are later checking for tty == NULL in __tty_open().

The following patch fixes the issue for me (Alan, feel free to merge
it back into your tty-fix-mismatch-in-lookup-han patch or replace by
a more complete __tty_open() reorganization):

From: Bartlomiej Zolnierkiewicz <bzolnier@...il.com>
Subject: [PATCH] tty: fix __tty_open()

On tty_driver_lookup_tty() failure __tty_open() should continue
instead of returning an error, fix it.

Also update tty_driver_lookup_tty() documentation while at it.

Signed-off-by: Bartlomiej Zolnierkiewicz <bzolnier@...il.com>
---
 drivers/char/tty_io.c |   10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

Index: b/drivers/char/tty_io.c
===================================================================
--- a/drivers/char/tty_io.c
+++ b/drivers/char/tty_io.c
@@ -1205,7 +1205,7 @@ static void tty_line_name(struct tty_dri
  *	@driver: the driver for the tty
  *	@idx:	 the minor number
  *
- *	Return the tty, if found or ERR_PTR() otherwise.
+ *	Return the tty, if found or NULL otherwise.
  *
  *	Locking: tty_mutex must be held. If tty is found, the mutex must
  *	be held until the 'fast-open' is also done. Will change once we
@@ -1813,16 +1813,10 @@ retry_open:
 		return -ENODEV;
 	}
 got_driver:
-	if (!tty) {
+	if (!tty)
 		/* check whether we're reopening an existing tty */
 		tty = tty_driver_lookup_tty(driver, inode, index);
 
-		if (tty == NULL) {
-			mutex_unlock(&tty_mutex);
-			return -ENODEV;
-		}
-	}
-
 	if (tty) {
 		retval = tty_reopen(tty);
 		if (retval)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/