linux-kernel - Re: [PATCH/RFC] VFS: support parallel updates in the one directory.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 24 Feb 2022 09:31:28 -0700
From:   Andreas Dilger <adilger@...ger.ca>
To:     NeilBrown <neilb@...e.de>
Cc:     "Darrick J. Wong" <djwong@...nel.org>,
        Dave Chinner <david@...morbit.com>,
        Al Viro <viro@...iv.linux.org.uk>,
        Linux NFS Mailing List <linux-nfs@...r.kernel.org>,
        linux-fsdevel@...r.kernel.org, LKML <linux-kernel@...r.kernel.org>,
        Daire Byrne <daire@...g.com>,
        Andreas Dilger <adilger.kernel@...ger.ca>
Subject: Re: [PATCH/RFC] VFS: support parallel updates in the one directory.

On Feb 23, 2022, at 22:57, NeilBrown <neilb@...e.de> wrote:
> 
> 
> I added this:
> --- a/fs/xfs/xfs_icache.c
> +++ b/fs/xfs/xfs_icache.c
> @@ -87,6 +87,7 @@ xfs_inode_alloc(
>    /* VFS doesn't initialise i_mode or i_state! */
>    VFS_I(ip)->i_mode = 0;
>    VFS_I(ip)->i_state = 0;
> +    VFS_I(ip)->i_flags |= S_PAR_UPDATE;
>    mapping_set_large_folios(VFS_I(ip)->i_mapping);
> 
>    XFS_STATS_INC(mp, vn_active);
> 
> and ran my highly sophisticated test in an XFS directory:
> 
> for i in {1..70}; do ( for j in {1000..8000}; do touch $j; rm -f $j ; done ) & done
> 
> This doesn't crash - which is a good sign.
> While that was going I tried
> while : ; do ls -l ; done
> 
> it sometimes reports garbage for the stat info:
> 
> total 0
> -????????? ? ?    ?    ?            ? 1749
> -????????? ? ?    ?    ?            ? 1764
> -????????? ? ?    ?    ?            ? 1765
> -rw-r--r-- 1 root root 0 Feb 24 16:47 1768
> -rw-r--r-- 1 root root 0 Feb 24 16:47 1770
> -rw-r--r-- 1 root root 0 Feb 24 16:47 1772
> ....
> 
> I *think* that is bad - probably the "garbage" that you referred to?
> 
> Obviously I gets lots of 
> ls: cannot access '1764': No such file or directory
> ls: cannot access '1749': No such file or directory
> ls: cannot access '1780': No such file or directory
> ls: cannot access '1765': No such file or directory
> 
> but that is normal and expected when you are creating and deleting
> files during the ls.

The "ls -l" output with "???" is exactly the case where the filename is
in readdir() but stat() on a file fails due to an unavoidable userspace 
race between the two syscalls and the concurrent unlink(). This is
probably visible even without the concurrent dirops patch. 

The list of affected filenames even correlates with the reported errors:
1764, 1765, 1769

It looks like everything is working as expected. 

Cheers, Andreas