linux-kernel - [PATCH 2/4] pty: Ignore slave pty close() if never successfully opened

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1359567832-25318-3-git-send-email-peter@hurleysoftware.com>
Date:	Wed, 30 Jan 2013 12:43:50 -0500
From:	Peter Hurley <peter@...leysoftware.com>
To:	Ilya Zykov <ilya@...x.ru>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>
Cc:	Alan Cox <alan@...ux.intel.com>, linux-serial@...r.kernel.org,
	linux-kernel@...r.kernel.org, Jiri Slaby <jslaby@...e.cz>,
	Peter Hurley <peter@...leysoftware.com>
Subject: [PATCH 2/4] pty: Ignore slave pty close() if never successfully opened

If the master and slave ptys are opened in parallel, the slave open
fails because the pty is still locked. This is as designed.
However, pty_close() is still called for the slave pty which sets
TTY_OTHER_CLOSED in the master pty. This can cause the master open
to fail as well.

Use a common pattern in other tty drivers by setting TTY_IO_ERROR
until the open is successful and only closing the pty if not set.

Note: the master pty always closes regardless of whether the open
was successful, so that proper cleanup can occur.

Signed-off-by: Peter Hurley <peter@...leysoftware.com>
---
 drivers/tty/pty.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/tty/pty.c b/drivers/tty/pty.c
index b4e5769..6eb3e80 100644
--- a/drivers/tty/pty.c
+++ b/drivers/tty/pty.c
@@ -38,9 +38,12 @@ static void pty_close(struct tty_struct *tty, struct file *filp)
 	if (tty->driver->subtype == PTY_TYPE_MASTER)
 		WARN_ON(tty->count > 1);
 	else {
+		if (test_bit(TTY_IO_ERROR, &tty->flags))
+			return;
 		if (tty->count > 2)
 			return;
 	}
+	set_bit(TTY_IO_ERROR, &tty->flags);
 	wake_up_interruptible(&tty->read_wait);
 	wake_up_interruptible(&tty->write_wait);
 	tty->packet = 0;
@@ -246,6 +249,8 @@ static int pty_open(struct tty_struct *tty, struct file *filp)
 	if (!tty || !tty->link)
 		goto out;
 
+	set_bit(TTY_IO_ERROR, &tty->flags);
+
 	retval = -EIO;
 	if (test_bit(TTY_OTHER_CLOSED, &tty->flags))
 		goto out;
@@ -254,6 +259,7 @@ static int pty_open(struct tty_struct *tty, struct file *filp)
 	if (tty->link->count != 1)
 		goto out;
 
+	clear_bit(TTY_IO_ERROR, &tty->flags);
 	clear_bit(TTY_OTHER_CLOSED, &tty->link->flags);
 	set_bit(TTY_THROTTLED, &tty->flags);
 	retval = 0;
-- 
1.8.1.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/