linux-kernel - Re: [PATCH v3 1/4] open: new O

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <vhq3osjqs3nn764wrp2lxp66b4dxpb3n5x3dijhe2yr53qfgy3@tfswbjskc3y6>
Date: Wed, 28 Jan 2026 08:12:07 +0100
From: Mateusz Guzik <mjguzik@...il.com>
To: Aleksa Sarai <cyphar@...har.com>
Cc: Dorjoy Chowdhury <dorjoychy111@...il.com>, 
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org, viro@...iv.linux.org.uk, 
	brauner@...nel.org, jack@...e.cz, jlayton@...nel.org, chuck.lever@...cle.com, 
	alex.aring@...il.com, arnd@...db.de, adilger@...ger.ca
Subject: Re: [PATCH v3 1/4] open: new O_REGULAR flag support

On Wed, Jan 28, 2026 at 12:23:45AM +0100, Aleksa Sarai wrote:
> In my view, this should be an openat2(2)-only API.

fwiw +1 from me, the O_ flag situation is already terrible even without
the validation woes.

I find it most unfortunate the openat2 syscall reuses the O_ namespace.
For my taste it would be best closed for business, with all new flag
additions using a different space.

I can easily see people passing O_WHATEVER to open and openat by blindly
assuming they are supported just based on the name.

that's a side mini-rant, too late to do anything here now

> In addition, I would
> propose that (instead of burning another O_* flag bit for this as a
> special-purpose API just for regular files) you could have a mask of
> which S_IFMT bits should be rejected as a new field in "struct
> open_how". This would let you reject sockets or device inodes but permit
> FIFOs and regular files or directories, for instance. This could even be
> done without a new O_* flag at all (the zero-value how->sfmt_mask would
> allow everything and so would work well with extensible structs), but we
> could add an O2_* flag anyway.

I don't think this works because the vars have overlapping bits:
  #define S_IFBLK  0060000
  #define S_IFDIR  0040000

So you very much can't select what you want off of a bitmask.

At best the field could be used to select the one type you are fine with.

If one was to pursue the idea, some other defines with unique bits would
need to be provided. But even then, semantics should be to only *allow*
the bits you are fine with and reject the rest.

But I'm not at all confident this is worth any effort -- with
O_DIRECTORY already being there and O_REGULAR proposed, is there a use
case which wants something else?

> 
> > +#define ENOTREG		134	/* Not a regular file */
> > +
> 
[..]
> Then to be fair, the existence of ENOTBLK, ENOTDIR, ENOTSOCK, etc. kind
> of justify the existence of ENOTREG too. Unfortunately, you won't be
> able to use ENOTREG if you go with my idea of having mask bits in
> open_how... (And what errno should we use then...? Hm.)
> 

The most useful behavior would indicate what was found (e.g., a pipe).

The easiest way to do it would create errnos for all types (EISDIR
already exists for one), but I can't seriously propose that.

Going the other way, EBADTYPE or something else reusable would be my
idea.