linux-ext4 - Re: fallocate(FALLOC_FL_ZERO_RANGE_BUT_REALLY) to avoid unwritten extents?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <c18d3d32-9504-016a-b7e7-feeddff0cde6@sandeen.net>
Date:   Mon, 4 Jan 2021 16:40:08 -0600
From:   Eric Sandeen <sandeen@...deen.net>
To:     Theodore Ts'o <tytso@....edu>, Andres Freund <andres@...razel.de>
Cc:     linux-fsdevel@...r.kernel.org, linux-xfs@...r.kernel.org,
        linux-ext4@...r.kernel.org, linux-block@...r.kernel.org
Subject: Re: fallocate(FALLOC_FL_ZERO_RANGE_BUT_REALLY) to avoid unwritten
 extents?

On 1/4/21 1:17 PM, Theodore Ts'o wrote:
> On Tue, Dec 29, 2020 at 10:28:19PM -0800, Andres Freund wrote:
>>
>> Would it make sense to add a variant of FALLOC_FL_ZERO_RANGE that
>> doesn't convert extents into unwritten extents, but instead uses
>> blkdev_issue_zeroout() if supported?  Mostly interested in xfs/ext4
>> myself, but ...
> 
> One thing to note is that there are some devices which support a write
> zeros operation, but where it is *less* performant than actually
> writing zeros via DMA'ing zero pages.  Yes, that's insane.
> Unfortunately, there are a insane devices out there....
> 
> This is not hypothetical; I know this because we tried using write
> zeros in mke2fs, and I got regression complaints where
> mke2fs/mkfs.ext4 got substantially slower for some devices.

Was this "libext2fs: mkfs.ext3 really slow on centos 8.2" ?

If so, wasn't the problem that it went from a few very large IOs to a
multitude of per-block fallocate calls, a problem which was fixed by
this commit?

commit 86d6153417ddaccbe3d1f4466a374716006581f4 (HEAD)
Author: Theodore Ts'o <tytso@....edu>
Date:   Sat Apr 25 11:41:24 2020 -0400

    libext2fs: batch calls to ext2fs_zero_blocks2()
    
    When allocating blocks for an indirect block mapped file, accumulate
    blocks to be zero'ed and then call ext2fs_zero_blocks2() to zero them
    in large chunks instead of block by block.
    
    This significantly speeds up mkfs.ext3 since we don't send a large
    number of ZERO_RANGE requests to the kernel, and while the kernel does
    batch write requests, it is not batching ZERO_RANGE requests.  It's
    more efficient to batch in userspace in any case, since it avoids
    unnecessary system calls.
    
    Reported-by: Mario Schuknecht <mario.schuknecht@...search-fe.de>
    Signed-off-by: Theodore Ts'o <tytso@....edu>


or do I have the wrong report above?

I ask because mkfs.xfs is now also using FALLOC_FL_ZERO_RANGE

Thanks,
-Eric