linux-ext4 - Re: Large directories and poor order correlation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4D7EA83D.20400@cfl.rr.com>
Date:	Mon, 14 Mar 2011 19:43:57 -0400
From:	Phillip Susi <psusi@....rr.com>
To:	Ted Ts'o <tytso@....edu>
CC:	Eric Sandeen <sandeen@...hat.com>,
	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: Large directories and poor order correlation

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/14/2011 05:52 PM, Ted Ts'o wrote:
> Unfortunately the kernel can't do it, because a directory could be
> arbitrarily big, and kernel memory is non-swappable.  In addition,

Buffers/cache is discardable though.  Or does the entire htree have to
be kept in slab or something?

> what if a process opens a directory, starts calling readdir, pauses in
> the middle, and then holds onto it for days, weeks, or months?

The same thing that happened before htree?

> It's not hard to provide library routines that do the right thing, and
> I have written an LD_PRELOAD which intercepts opendir() and readdir()
> calls and does the sorting in userspace.  Perhaps the right answer is
> getting this into libc, but I have exactly two words for you: "Uhlrich
> Drepper".

Wouldn't it be better to just have readdir() use the main directory,
which presumably is in a more sane ordering, instead of the htree?  That
way you don't have to burn cpu and ram sorting on every opendir().

Also, I have checked some smaller directories and lsattr reports they
are NOT using indexing, yet still display poor correlation.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1+qDkACgkQJ4UciIs+XuIktwCgi1u4T2x+igOw4feO0hNjzB9W
liIAmwRBdPiZMSfWpzu4+40xJsNXzouQ
=d4VX
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html