linux-kernel - Re: [regression] ENOTTY returned for tty fds

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <2024072401-obtain-heap-6d8d@gregkh>
Date: Wed, 24 Jul 2024 12:15:39 +0200
From: Greg KH <gregkh@...uxfoundation.org>
To: stsp <stsp2@...dex.ru>
Cc: linux-serial@...r.kernel.org,
	Linux kernel <linux-kernel@...r.kernel.org>,
	Herbert Xu <herbert@...dor.apana.org.au>,
	Johan Hovold <johan@...nel.org>
Subject: Re: [regression] ENOTTY returned for tty fds

On Wed, Jul 24, 2024 at 12:55:22PM +0300, stsp wrote:
> 24.07.2024 12:08, Greg KH пишет:
> > On Wed, Jul 24, 2024 at 11:07:32AM +0300, stsp wrote:
> > > 24.07.2024 09:51, Greg KH пишет:
> > > > What caused this change/regression?
> > > I have absolutely no idea.
> > > I've found it by debugging userspace,
> > > and wrote a test-case to make sure the
> > > problem is not in user-space.
> > So this has always worked this way?  Or has it changed?  If changed,
> > when did it work before?
> 
> OK, I did some extensive digging, and
> now am pretty sure its this one:
> 
> commit 1b8b20868a6d64cfe8174a21b25b74367bdf0560
> Author:     Johan Hovold <johan@...nel.org>
> AuthorDate: Wed Apr 7 11:52:02 2021 +0200
> Commit:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>
> CommitDate: Sat Apr 10 10:36:34 2021 +0200
> 
>     tty: fix return value for unsupported ioctls
> 
>     Drivers should return -ENOTTY ("Inappropriate I/O control operation")
>     when an ioctl isn't supported, while -EINVAL is used for invalid
>     arguments.
> 
> > > >     And does any real-world programs
> > > > rely on this?
> > > dosemu
> > It does this today or wants to do this in the future?
> 
> It does so since 2003/03/15, according
> to a change-log (so for 21 year now).
> Adding Herbert Xu to CC as an author of
> that feature.
> 
> > > >     What exactly are you trying to determine with this ioctl
> > > > test?
> > > Whether it is a PTS (Pseudo-Tty-Slave), or
> > > a real comport with MSR signalling.
> > Why is that needed?
> 
> To fake modem status lines, mustly DCD.
> 
> 
> >    And why not do it how other programs (like stty)
> > does it?
> 
> I am not sure stty is interested in faking
> modem status lines. It doesn't seem to be
> using TIOCM ioctls at all.
> 
> > > >     Is there a different way to determine that?
> > > I am not aware of any "canonical" way
> > > of determining this. Maybe you tell me. :)
> > > So far the only fix I know, is to stop checking
> > > errno. But you return ENOTTY for a tty-associated
> > > fd (isatty(fd)==1), so I believe this is a
> > > bug in a kernel.
> > isatty() is a libc provided function, not a kernel call.
> It seems to be using TCGETS ioctl() and looks
> for it to not return an error.
> I still think returning ENOTTY where TCGETS
> succeeds, is more than strange.
> I looked at the tty code of current linux kernel,
> and it returns EOPNOTSUPP in most such cases.
> At least that makes sense, but ENOTTY?

-ENOTTY is the documented result of invalid ioctl arguments sent, I am
pretty sure POSIX requires this somewhere.  So this was fixing a
requirement here...

thanks,

greg k-h