linux-ext4 - Re: fallocate(FALLOC_FL_ZERO_RANGE_BUT_REALLY) to avoid unwritten extents?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 4 Jan 2021 11:10:58 -0800
From:   Andres Freund <andres@...razel.de>
To:     "Darrick J. Wong" <darrick.wong@...cle.com>
Cc:     linux-fsdevel@...r.kernel.org, linux-xfs@...r.kernel.org,
        linux-ext4@...r.kernel.org, linux-block@...r.kernel.org
Subject: Re: fallocate(FALLOC_FL_ZERO_RANGE_BUT_REALLY) to avoid unwritten
 extents?

Hi,

On 2021-01-04 10:19:58 -0800, Darrick J. Wong wrote:
> On Tue, Dec 29, 2020 at 10:28:19PM -0800, Andres Freund wrote:
> > Would it make sense to add a variant of FALLOC_FL_ZERO_RANGE that
> > doesn't convert extents into unwritten extents, but instead uses
> > blkdev_issue_zeroout() if supported?  Mostly interested in xfs/ext4
> > myself, but ...
> > 
> > Doing so as a variant of FALLOC_FL_ZERO_RANGE seems to make the most
> > sense, as that'd work reasonably efficiently to initialize newly
> > allocated space as well as for zeroing out previously used file space.
> > 
> > 
> > As blkdev_issue_zeroout() already has a fallback path it seems this
> > should be doable without too much concern for which devices have write
> > zeroes, and which do not?
> 
> Question: do you want the kernel to write zeroes even for devices that
> don't support accelerated zeroing?

I don't have a strong opinion on it. A complex userland application can
do a bit better job managing queue depth etc, but otherwise I suspect
doing the IO from kernel will win by a small bit. And the queue-depth
issue presumably would be relevant for write-zeroes as well, making me
lean towards just using the fallback.


> Since I assume that if the fallocate fails you'll fall back to writing
> zeroes from userspace anyway...

And there's non-linux platforms as well, at least that's the rumor I hear.


> Second question: Would it help to have a FALLOC_FL_DRY_RUN flag that
> could be used to probe if a file supports fallocate without actually
> changing anything?  I'm (separately) pursuing a fix for the loop device
> not being able to figure out if a file actually supports a particular
> fallocate mode.

Hm. I can see some potential uses of such a flag, but I haven't really
wished for it so far.

Greetings,

Andres Freund