lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Thu, 08 Mar 2012 12:02:07 -0500
From:	Phillip Susi <psusi@...ntu.com>
To:	Theodore Tso <tytso@....EDU>
CC:	Jacek Luczak <difrost.kernel@...il.com>,
	linux-ext4@...r.kernel.org,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>,
	LKML <linux-kernel@...r.kernel.org>, linux-btrfs@...r.kernel.org
Subject: Re: getdents - ext4 vs btrfs performance

On 2/29/2012 11:44 PM, Theodore Tso wrote:
> You might try sorting the entries returned by readdir by inode number
> before you stat them.    This is a long-standing weakness in
> ext3/ext4, and it has to do with how we added hashed tree indexes to
> directories in (a) a backwards compatible way, that (b) was POSIX
> compliant with respect to adding and removing directory entries
> concurrently with reading all of the directory entries using
> readdir.

When I ran into this a while back, I cobbled together this python script 
to measure the correlation from name to inode, inode to first data 
block, and name to first data block for all of the small files in a 
large directory, and found that ext4 gives a very poor correlation due 
to that directory hashing.  This is one of the reasons I prefer using 
dump instead of tar for backups, since it rips through my Maildir more 
than 10 times faster than tar, since it reads the files in inode order.


View attachment "correlation.py" of type "text/plain" (1391 bytes)

Powered by blists - more mailing lists