linux-kernel - Re: regression, bisected: openpty fails from 3.7 onwards without devpts

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <50EEF8FD.9000004@suse.cz>
Date:	Thu, 10 Jan 2013 18:23:09 +0100
From:	Jiri Slaby <jslaby@...e.cz>
To:	Florian Westphal <fw@...len.de>
CC:	Alan Cox <alan@...ux.intel.com>, linux-kernel@...r.kernel.org,
	gaowanlong@...fujitsu.com, gregkh@...uxfoundation.org,
	frank@...htenheld.de
Subject: Re: regression, bisected: openpty fails from 3.7 onwards without
 devpts

On 01/10/2013 05:29 PM, Florian Westphal wrote:
> Alan Cox <alan@...ux.intel.com> wrote:
>> > On Thu, 10 Jan 2013 15:46:26 +0100
>> > Florian Westphal <fw@...len.de> wrote:
>>> > > Frank Lichtenheld discovered that openpty() doesn't work anymore when
>>> > > /dev/pts is not present.
>>> > > 
>>> > > We bisected this down to
>>> > > 
>>> > > commit bbb63c514a3464342967237a51a21ea8f61ab951
>>> > > Author: Wanlong Gao <gaowanlong@...fujitsu.com>
>>> > > Subject: drivers:tty:fix up ENOIOCTLCMD error handling
> [..]
> 
>>> > > #include <stdio.h>
>>> > > #include <pty.h>
>>> > > int main(void) {
>>> > >  int pty_fd, tty_fd;
>>> > >  if (openpty(&pty_fd, &tty_fd, NULL, NULL, NULL) != 0) {
>>> > >   perror("openpty");
>>> > >   return 1;
>>> > >  }
>>> > > return 0;
>>> > > }
>>> > > ----
>>> > > [ compile with cc -lutil pty.c -o pty ]
>>> > > 
>>> > > If devpts is available or above commit reverted openpty works again.
>> > 
>> > The commit is fairly general - what we need to do here is to figure out
>> > which specific thing trips up openpty so we can put the error on that
>> > back as it was (or find a better way) so it still works.
>> > 
>> > Can you attach an strace of the working/failing cases without /dev/pts 
> Sure, attached.  /dev/pts is not present.
> Both traces are from the same machine, with same kernel version (except
> above commit reverted).

> execve("./ptytest", ["./ptytest"], [/* 7 vars */]) = 0
...
> open("/dev/ptmx", O_RDWR)         = -1 ENOSPC (No space left on device)
> open("/dev/ptyp0", O_RDWR)        = 3
> ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
> ioctl(3, TIOCGPTN, [0])           = -1 ENOTTY (Inappropriate ioctl for device)


getptsname expects EINVAL on failure to fall back to /dev/ttyp*... The
same as unlockpt. We should definitely revert now and can teach glibc to
accept also ENOTTY. After some years, we can try again :).

thanks,
-- 
js
suse labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/