linux-ext4 - Re: Question on readdir implementation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090915183849.GB15101@mit.edu>
Date:	Tue, 15 Sep 2009 14:38:49 -0400
From:	Theodore Tso <tytso@....edu>
To:	Florian Weimer <fw@...eb.enyo.de>
Cc:	Zhang Huan <zhhuan@...il.com>, linux-ext4@...r.kernel.org
Subject: Re: Question on readdir implementation

On Tue, Sep 15, 2009 at 05:56:32PM +0000, Florian Weimer wrote:
> * Theodore Tso:
> 
> > So this is something we could do in the future.  In practice, no one
> > has complained about this as far as NFS is concerned, so it's not high
> > priority for me to pursue.  Were you worried about this as a practical
> > matter, or as a theoretical one?
> 
> readdir returning entries in essentially randomized order is a
> practical performance problem for many things, from grep -r to tar. 8-(
> (My recent FIBMAP/FIEMAP question was related to that, too.)

Well, it's not _that_ hard for applications to sort the directory
entries by inode number.  I've written a LD_PRELOAD, called
spd_readdir() for people who want to use it.  The mutt application
does this natively, and it makes the problem go away.

We could do this in the kernel, but for very large directories, you
will end up pinning large amounts of memory --- and if an application
holds a directory fd open for a long time, the memory needs to be kept
pinned until the dfd is closed.  Still, for moderate sized
directories, it's a possibility.

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html