linux-kernel - Re: [musl] getdents64 lost direntries with SMB/NFS and buffer size

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <8736eiqq1f.fsf@mid.deneb.enyo.de>
Date:   Wed, 20 Nov 2019 20:57:32 +0100
From:   Florian Weimer <fw@...eb.enyo.de>
To:     Rich Felker <dalias@...c.org>
Cc:     linux-fsdevel@...r.kernel.org, musl@...ts.openwall.com,
        linux-kernel@...r.kernel.org, linux-nfs@...r.kernel.org,
        linux-cifs@...r.kernel.org
Subject: Re: [musl] getdents64 lost direntries with SMB/NFS and buffer size < unknown threshold

* Rich Felker:

> An issue was reported today on the Alpine Linux tracker at
> https://gitlab.alpinelinux.org/alpine/aports/issues/10960 regarding
> readdir results from SMB/NFS shares with musl libc.
>
> After a good deal of analysis, we determined the root cause to be that
> the second and subsequent calls to getdents64 are dropping/skipping
> direntries (that have not yet been deleted) when some entries were
> deleted following the previous call. The issue appears to happen only
> when the buffer size passed to getdents64 is below some threshold
> greater than 2k (the size musl uses) but less than 32k (the size glibc
> uses, with which we were unable to reproduce the issue).

>From the Gitlab issue:

  while ((dp = readdir(dir)) != NULL) {
      unlink(dp->d_name);
      ++file_cnt;
  }

I'm not sure that this is valid code to delete the contents of a
directory.  It's true that POSIX says this:

| If a file is removed from or added to the directory after the most
| recent call to opendir() or rewinddir(), whether a subsequent call
| to readdir() returns an entry for that file is unspecified.

But many file systems simply provide not the necessary on-disk data
structures which are need to ensure stable iteration in the face of
modification of the directory.  There are hacks, of course, such as
compacting the on-disk directory only on file creation, which solves
the file removal case.

For deleting an entire directory, that is not really a problem because
you can stick another loop around this while loop which re-reads the
directory after rewinddir.  Eventually, it will become empty.