linux-ext4 - Re: unexpected sync delays in dpkg for small pre-allocated files on ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160526070217.GA12151@infradead.org>
Date:	Thu, 26 May 2016 00:02:17 -0700
From:	Christoph Hellwig <hch@...radead.org>
To:	Dave Chinner <david@...morbit.com>
Cc:	Gernot Hillier <gernot.hillier@...mens.com>,
	linux-ext4@...r.kernel.org, guillem@...ian.org
Subject: Re: unexpected sync delays in dpkg for small pre-allocated files on
 ext4

On Thu, May 26, 2016 at 12:20:20PM +1000, Dave Chinner wrote:
> Stupid question: why is dpkg using fallocate() for such small ranges
> like that? I can't think of a more inefficient way to do small IO -
> using delayed allocation is far more optimal from a layout,
> overhead, latency and IO perspective than the above forced
> allocation pseudo-synchronous write behaviour.

Below is the commit adding it.  Guillem, can you explain what fallocate
in the dpkg is supposed to help with?  The way it's it does fallocate
before writes that aren't spare it looks actively harmful for any 
Linux file system.

commit 87b0b20b86407baf1deb4e91b3fd839e01228ac8
Author: Guillem Jover <guillem@...ian.org>
Date:   Tue Jul 15 21:00:52 2014 +0200

    dpkg: Try to preallocate the disk size for extracted files
        
    This might help in avoiding filesystem fragmentation, and possibly
    improve performance on some filesystems.
		    
    We use the system specific methods if available, and possibly
    improve performance on some filesystems.

    We use the system specific methods if available, and only use
    posix_fallocate() if nothing else is available, because on some systems
    its semantics are counter to what we want to obtain here, as the libc
    library will fallback to manually writing '\0' to each block to force
    the preallocation, instead of just failing and leaving the application
    to do so if desired.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html