linux-ext4 - Re: Problems with determining data presence by examining extents?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 15 Jan 2020 09:38:54 +0100
From:   Christoph Hellwig <hch@....de>
To:     David Howells <dhowells@...hat.com>
Cc:     linux-fsdevel@...r.kernel.org, viro@...iv.linux.org.uk, hch@....de,
        tytso@....edu, adilger.kernel@...ger.ca, darrick.wong@...cle.com,
        clm@...com, josef@...icpanda.com, dsterba@...e.com,
        linux-ext4@...r.kernel.org, linux-xfs@...r.kernel.org,
        linux-btrfs@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: Problems with determining data presence by examining extents?

On Tue, Jan 14, 2020 at 04:48:29PM +0000, David Howells wrote:
> Again with regard to my rewrite of fscache and cachefiles:
> 
> 	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=fscache-iter
> 
> I've got rid of my use of bmap()!  Hooray!
> 
> However, I'm informed that I can't trust the extent map of a backing file to
> tell me accurately whether content exists in a file because:
> 
>  (a) Not-quite-contiguous extents may be joined by insertion of blocks of
>      zeros by the filesystem optimising itself.  This would give me a false
>      positive when trying to detect the presence of data.
> 
>  (b) Blocks of zeros that I write into the file may get punched out by
>      filesystem optimisation since a read back would be expected to read zeros
>      there anyway, provided it's below the EOF.  This would give me a false
>      negative.

The whole idea of an out of band interface is going to be racy and suffer
from implementation loss.  I think what you want is something similar to
the NFSv4.2 READ_PLUS operation - give me that if there is any and
otherwise tell me that there is a hole.  I think this could be a new
RWF_NOHOLE or similar flag, just how to return the hole size would be
a little awkward.  Maybe return a specific negative error code (ENODATA?)
and advance the iov anyway.