linux-kernel - Re: splice(-> FIFO) never wakes up inotify IN

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAOQ4uxjizutWR37dm5RxiBY_L-bVHndJYaK_CHi88ZTT0DNpjg@mail.gmail.com>
Date:   Mon, 26 Jun 2023 19:21:16 +0300
From:   Amir Goldstein <amir73il@...il.com>
To:     Ahelenia Ziemiańska 
        <nabijaczleweli@...ijaczleweli.xyz>
Cc:     Alexander Viro <viro@...iv.linux.org.uk>,
        Christian Brauner <brauner@...nel.org>,
        linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
        Jan Kara <jack@...e.cz>,
        Chung-Chiang Cheng <cccheng@...ology.com>
Subject: Re: splice(-> FIFO) never wakes up inotify IN_MODIFY?

On Mon, Jun 26, 2023 at 6:12 PM Ahelenia Ziemiańska
<nabijaczleweli@...ijaczleweli.xyz> wrote:
>
> On Mon, Jun 26, 2023 at 05:53:46PM +0300, Amir Goldstein wrote:
> > > So is it really true that the only way to poll a pipe is a
> > > sleep()/read(O_NONBLOCK) loop?
> > I don't think so, but inotify is not the way.
> So what is? What do the kernel developers recommend as a way to see if a
> file is written to, and that file happens to be a pipe?
>
> FTR, I've opened the symmetric Debian#1039488:
>   https://bugs.debian.org/1039488
> against coreutils, since, if this is expected, and writing to a pipe
> should not generate write events on that pipe, then tail -f is currently
> broken on most systems.

First of all, it is better to mention that you are trying to fix a real
world use case when you are reporting a kernel misbehavior.

What this makes me wonder is, if tail -f <fifo> is broken as you claim
it is, how is it that decades go by without anyone noticing this problem?

When looking at tail source code I see:

/* Mark as '.ignore'd each member of F that corresponds to a
   pipe or fifo, and return the number of non-ignored members.  */
static size_t
ignore_fifo_and_pipe (struct File_spec *f, size_t n_files)
{
  /* When there is no FILE operand and stdin is a pipe or FIFO
     POSIX requires that tail ignore the -f option.
     Since we allow multiple FILE operands, we extend that to say: with -f,
     ignore any "-" operand that corresponds to a pipe or FIFO.  */

and it looks like tail_forever_inotify() is not being called unless
there are non pipes:

  if (forever && ignore_fifo_and_pipe (F, n_files))
    {

The semantics of tail -f on a pipe input would be very odd, because
the writer would need to close before tail can figure out which are
the last lines.

So honestly, we could maybe add IN_ACCESS/IN_MODIFY for the
splice_pipe_to_pipe() case, but I would really like to know what
the real use case is.

Another observation is that splice(2) never used to report any
inotify events at all until a very recent commit in v6.4
983652c69199 ("splice: report related fsnotify events")
but this commit left out the splice_pipe_to_pipe() case.

CC the author of the patch to ask why this case was left
out and whether he would be interested in fixing that.

Thanks,
Amir.