linux-kernel - Re: Problems with determining data presence by examining extents?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <7233E240-8EE5-4CD1-B8A4-A90925F51A1B@dilger.ca>
Date:   Wed, 15 Jan 2020 15:11:51 -0700
From:   Andreas Dilger <adilger@...ger.ca>
To:     David Howells <dhowells@...hat.com>
Cc:     Christoph Hellwig <hch@....de>, Qu Wenruo <quwenruo.btrfs@....com>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Al Viro <viro@...iv.linux.org.uk>,
        "Theodore Y. Ts'o" <tytso@....edu>,
        "Darrick J. Wong" <darrick.wong@...cle.com>,
        Chris Mason <clm@...com>, Josef Bacik <josef@...icpanda.com>,
        David Sterba <dsterba@...e.com>,
        linux-ext4 <linux-ext4@...r.kernel.org>,
        linux-xfs <linux-xfs@...r.kernel.org>,
        linux-btrfs <linux-btrfs@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: Problems with determining data presence by examining extents?

On Jan 15, 2020, at 1:55 PM, David Howells <dhowells@...hat.com> wrote:
> 
> Andreas Dilger <adilger@...ger.ca> wrote:
> 
>> I think what is needed here is an fadvise/ioctl that tells the filesystem
>> "don't allocate blocks unless actually written" for that file.
> 
> Yeah - and it would probably need to find its way onto disk so that its effect
> is persistent and visible to out-of-kernel tools.
> 
> It would also have to say that blocks of zeros shouldn't be optimised away.

I don't necessarily see that as a requirement, so long as the filesystem
stores a "block" at that offset, but it could dedupe all zero-filled blocks
to the same "zero block".  That still allows saving storage space, while
keeping the semantics of "this block was written into the file" rather than
"there is a hole at this offset".

Cheers, Andreas






Download attachment "signature.asc" of type "application/pgp-signature" (874 bytes)