linux-ext4 - Re: [PATCH] ext4: add noorlov parameter to avoid spreading of directory inodes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Wed, 2 Oct 2013 19:09:16 +0200 (CEST)
From:	Lukáš Czerner <lczerner@...hat.com>
To:	Benjamin LaHaise <bcrl@...ck.org>
cc:	Eric Sandeen <sandeen@...hat.com>, Jan Kara <jack@...e.cz>,
	"Theodore Ts'o" <tytso@....edu>,
	Andreas Dilger <adilger.kernel@...ger.ca>,
	linux-ext4@...r.kernel.org
Subject: Re: [PATCH] ext4: add noorlov parameter to avoid spreading of
 directory inodes

On Wed, 2 Oct 2013, Benjamin LaHaise wrote:

> Date: Wed, 2 Oct 2013 12:52:10 -0400
> From: Benjamin LaHaise <bcrl@...ck.org>
> To: Lukáš Czerner <lczerner@...hat.com>
> Cc: Eric Sandeen <sandeen@...hat.com>, Jan Kara <jack@...e.cz>,
>     Theodore Ts'o <tytso@....edu>, Andreas Dilger <adilger.kernel@...ger.ca>,
>     linux-ext4@...r.kernel.org
> Subject: Re: [PATCH] ext4: add noorlov parameter to avoid spreading of
>     directory inodes
> 
> On Wed, Oct 02, 2013 at 06:44:56PM +0200, Lukáš Czerner wrote:
> > What is the frequency of unlink operation in comparison to file
> > creation ? There is a possible issue with the global goal cursors
> > s_mb_last_group and s_mb_last_start which might make your files
> > increasingly scattered across the disk. I've attempted to address
> > this problem with my patch
> 
> unlink() is 1:1 with creation.  The storage on disk is essentially used 
> as an elastic buffer for the rest of the system.  That said, unlink() may 
> not occur for minutes, hours or days.
> 
> In terms of actual allocation pattern on disk, using fallocate() is 
> sufficient to ensure that files on disk are usually limited to 1 extent, 
> and allocated immediately following each other.  Without fallocate(), I 
> was seeing allocations aligned to 2048 block boundaries for 9MB files, 
> which hurt performance quite a bit.

I was not talking about internal file fragmentation but the actual
files lying further from the ideal block group they should have been
laid out.

> 
> > ext4: Try to better reuse recently freed space
> > 
> > What is the usual size of the files this application is creating ?
> 
> It varies.  The target is 8MB, but it ranges from 5MB to 22MB.  In the 
> worst case it can be as small as 4KB.

It means that you're indeed using stream allocation so this issue
would affect you.

> 
> 		-ben
> 
> > Thanks!
> > -Lukas
> 
>