linux-kernel - Re: [PATCH v3 1/4] open: new O

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <2026-01-28-content-sandy-strife-cartels-hShKtl@cyphar.com>
Date: Wed, 28 Jan 2026 20:26:48 +0100
From: Aleksa Sarai <cyphar@...har.com>
To: Mateusz Guzik <mjguzik@...il.com>
Cc: Dorjoy Chowdhury <dorjoychy111@...il.com>, 
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org, viro@...iv.linux.org.uk, 
	brauner@...nel.org, jack@...e.cz, jlayton@...nel.org, chuck.lever@...cle.com, 
	alex.aring@...il.com, arnd@...db.de, adilger@...ger.ca
Subject: Re: [PATCH v3 1/4] open: new O_REGULAR flag support

On 2026-01-28, Mateusz Guzik <mjguzik@...il.com> wrote:
> On Wed, Jan 28, 2026 at 12:23:45AM +0100, Aleksa Sarai wrote:
> > In my view, this should be an openat2(2)-only API.
> 
> fwiw +1 from me, the O_ flag situation is already terrible even without
> the validation woes.
> 
> I find it most unfortunate the openat2 syscall reuses the O_ namespace.
> For my taste it would be best closed for business, with all new flag
> additions using a different space.

We don't have any openat2(2)-only O_* flags yet, I agree that new flag
additions (except for very rare cases where you can make them backward
compatible -- such as a hypothetical O_EMPTYPATH) should be O2_* or
OEXT_* or something.

> I can easily see people passing O_WHATEVER to open and openat by blindly
> assuming they are supported just based on the name.

Yeah, if we don't do that it'll lead to confusion. openat2(2) has
exclusive rights to the 64-bit flag bits so we could start with those
before we need to cross with the O_* flag space.

> that's a side mini-rant, too late to do anything here now
> 
> > In addition, I would
> > propose that (instead of burning another O_* flag bit for this as a
> > special-purpose API just for regular files) you could have a mask of
> > which S_IFMT bits should be rejected as a new field in "struct
> > open_how". This would let you reject sockets or device inodes but permit
> > FIFOs and regular files or directories, for instance. This could even be
> > done without a new O_* flag at all (the zero-value how->sfmt_mask would
> > allow everything and so would work well with extensible structs), but we
> > could add an O2_* flag anyway.
> 
> I don't think this works because the vars have overlapping bits:
>   #define S_IFBLK  0060000
>   #define S_IFDIR  0040000
> 
> So you very much can't select what you want off of a bitmask.

Well, you can filter on S_IFCHR if you want to block both block/char
devices, but yeah the overlap is quite unfortunate... (That would also
mean blocking directories would also block S_IFBLK -- I remembered there
was an overlap but I forgot it coincided with S_IFDIR... Damn wacky
APIs.)

> At best the field could be used to select the one type you are fine with.
> 
> If one was to pursue the idea, some other defines with unique bits would
> need to be provided. But even then, semantics should be to only *allow*
> the bits you are fine with and reject the rest.
> 
> But I'm not at all confident this is worth any effort -- with
> O_DIRECTORY already being there and O_REGULAR proposed, is there a use
> case which wants something else?

There's also O_NOFOLLOW in a similar vein.

I can see someone wanting to permit FIFOs, regular files, and
directories being fine but blocking everything else. None of O_REGULAR,
O_DIRECTORY, nor O_NOFOLLOW provide that.

> > > +#define ENOTREG		134	/* Not a regular file */
> > > +
> > 
> [..]
> > Then to be fair, the existence of ENOTBLK, ENOTDIR, ENOTSOCK, etc. kind
> > of justify the existence of ENOTREG too. Unfortunately, you won't be
> > able to use ENOTREG if you go with my idea of having mask bits in
> > open_how... (And what errno should we use then...? Hm.)
> > 
> 
> The most useful behavior would indicate what was found (e.g., a pipe).
> 
> The easiest way to do it would create errnos for all types (EISDIR
> already exists for one), but I can't seriously propose that.

It might be kinda neat from a potential re-use perspective in other APIs
but yeah it would be quite wasteful to burn 3-5 errnos for this when we
already have ~4 that are logical inverses.

> Going the other way, EBADTYPE or something else reusable would be my
> idea.

I think that would be reasonable and if you word the error message
carefully you can even see it being a fairly generic errno for other
places to use.

-- 
Aleksa Sarai
https://www.cyphar.com/

Download attachment "signature.asc" of type "application/pgp-signature" (266 bytes)