linux-ext4 - Re: fallocate(FALLOC_FL_ZERO_RANGE_BUT_REALLY) to avoid unwritten extents?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <6D9D9B4D-65E5-4993-AC08-080B677BA78E@dilger.ca>
Date:   Mon, 18 Jan 2021 20:44:04 -0700
From:   Andreas Dilger <adilger@...ger.ca>
To:     Avi Kivity <avi@...lladb.com>
Cc:     Andres Freund <andres@...razel.de>,
        "Darrick J. Wong" <darrick.wong@...cle.com>,
        Christoph Hellwig <hch@...radead.org>,
        linux-fsdevel@...r.kernel.org, linux-xfs@...r.kernel.org,
        linux-ext4@...r.kernel.org, linux-block@...r.kernel.org
Subject: Re: fallocate(FALLOC_FL_ZERO_RANGE_BUT_REALLY) to avoid unwritten
 extents?

On Jan 13, 2021, at 12:44 AM, Avi Kivity <avi@...lladb.com> wrote:
> 
> On 1/12/21 11:36 PM, Andres Freund wrote:
>> Hi,
>> 
>> On 2021-01-12 13:14:45 -0800, Darrick J. Wong wrote:
>>> ALLOCSP64 can only allocate pre-zeroed blocks as part of extending EOF,
>>> whereas a new FZERO flag means that we can pre-zero an arbitrary range
>>> of bytes in a file.  I don't know if Avi or Andres' usecases demand that
>>> kind of flexibilty but I know I'd rather go for the more powerful
>>> interface.
>> Postgres/I don't at the moment have a need to allocate "written" zeroed
>> space anywhere but EOF. I can see some potential uses for more flexible
>> pre-zeroing in the future though, but not very near term.
>> 
> 
> I also agree that it's better not to have the kernel fall back internally on writing zeros, letting userspace do that. The assumption is that WRITE SAME will be O(1)-ish and so can bypass scheduling decisions, but if we need to write zeros, better let the application throttle the rate.

Writing zeroes from userspace has a *lot* more overhead when there is a network
filesystem involved.  It would be better to generate the zeroes on the server,
or directly in the disk than sending GB of zeroes over the network.


Cheers, Andreas






Download attachment "signature.asc" of type "application/pgp-signature" (874 bytes)