linux-kernel - Re: RFC: Serial port DTR/RTS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <f643f1f6-7e69-4be6-ac8a-7b1a3a9c402d@zytor.com>
Date: Tue, 18 Nov 2025 10:05:47 -0800
From: "H. Peter Anvin" <hpa@...or.com>
To: Ned Ulbricht <nedu@...scape.net>, "Maciej W. Rozycki" <macro@...am.me.uk>
Cc: Greg KH <gregkh@...uxfoundation.org>, "Theodore Ts'o" <tytso@....edu>,
        Maarten Brock <Maarten.Brock@...ls.nl>,
        "linux-serial@...r.kernel.org" <linux-serial@...r.kernel.org>,
        "linux-api@...r.kernel.org" <linux-api@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: RFC: Serial port DTR/RTS - O_<something>

On 2025-11-18 08:33, Ned Ulbricht wrote:
>>
>> O_NOCLOBBER looks like an odd in-between between O_EXCL and
>> (O_EXCL|O_NOFOLLOW); stated to be specifically to implement the shell
>> "noclobber" semantic.
> 
> "(O_EXCL|O_NOFOLLOW)" provokes a thought...
> 
> As essential context, fs/open.c build_open_flags() has:
> 
> if (flags & O_CREAT) {
>     op->intent |= LOOKUP_CREATE;
>     if (flags & O_EXCL) {
>         op->intent |= LOOKUP_EXCL;
>         flags |= O_NOFOLLOW;
>     }
> }
> 
> if (!(flags & O_NOFOLLOW))
>     lookup_flags |= LOOKUP_FOLLOW;
> 

Interesting. As far as O_NOCLOBBER is concerned, that is an "O_EXCL unless the
output is a special file (device node, FIFO, etc)"; presumably to allow the
shell to not flip out when doing, say "foo > /dev/ttyS0" when in noclobber mode.

I had missed the bit in the spec that says that O_CREAT|O_EXCL is required to
imply O_NOFOLLOW (as Linux indeed does as seen above.)

O_NOCLOBBER emulation in user space would seem to be possible with a loop;
first try to open O_CREAT|O_EXCL and if that fails with EEXIST then open
without either; if that succeeds test with fstat() to see if it is a regular
file, and if it is, close it and error. However, it is hardly ideal, and I
might have overlooked some mechanism by which this may fail.

	-hpa