linux-ext4 - Re: Problems with determining data presence by examining extents?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <24870.1579098019@warthog.procyon.org.uk>
Date:   Wed, 15 Jan 2020 14:20:19 +0000
From:   David Howells <dhowells@...hat.com>
To:     Andreas Dilger <adilger@...ger.ca>
Cc:     dhowells@...hat.com, Qu Wenruo <quwenruo.btrfs@....com>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Al Viro <viro@...iv.linux.org.uk>,
        Christoph Hellwig <hch@....de>,
        "Theodore Y. Ts'o" <tytso@....edu>,
        "Darrick J. Wong" <darrick.wong@...cle.com>,
        Chris Mason <clm@...com>, Josef Bacik <josef@...icpanda.com>,
        David Sterba <dsterba@...e.com>,
        linux-ext4 <linux-ext4@...r.kernel.org>,
        linux-xfs <linux-xfs@...r.kernel.org>,
        linux-btrfs <linux-btrfs@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: Problems with determining data presence by examining extents?

Andreas Dilger <adilger@...ger.ca> wrote:

> > Would you like to explain why you want to know such fs internal info?
> 
> I believe David wants it to store sparse files as an cache and use FIEMAP to
> determine if the blocks are cached locally, or if they need to be fetched from
> the server.  If the filesystem doesn't store the written blocks accurately,
> there is no way for the local cache to know whether it is holding valid data
> or not.

More or less.  I have no particular attachment to bmap or FIEMAP as the
interface to use.  I'm just interested in finding out quickly if the data I
want is present.

If call_read_iter() will return a short read on hitting a hole, I can manage
if I can find out if just the first byte is present.

Finding out if the block is present allows me to avoid shaping read requests
from VM readahead into 256k blocks - which may require the allocation of extra
pages for bufferage.

David