linux-kernel - Re: very poor ext3 write performance on big filesystems?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <47BB2008.6070202@rtr.ca>
Date:	Tue, 19 Feb 2008 13:29:28 -0500
From:	Mark Lord <lkml@....ca>
To:	Theodore Tso <tytso@....edu>, Andi Kleen <andi@...stfloor.org>,
	Tomasz Chmielewski <mangoo@...g.org>,
	LKML <linux-fsdevel@...r.kernel.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: very poor ext3 write performance on big filesystems?

Theodore Tso wrote:
..
> The following ld_preload can help in some cases.  Mutt has this hack
> encoded in for maildir directories, which helps.
..

Oddly enough, that same spd_readdir() preload craps out here too
when used with "rm -r" on largish directories.

I added a bit more debugging to it, and it always craps out like this:

     opendir dir=0x805ad10((nil))
     Readdir64 dir=0x805ad10 pos=0/289/290
     Readdir64 dir=0x805ad10 pos=1/289/290
     Readdir64 dir=0x805ad10 pos=2/289/290
     Readdir64 dir=0x805ad10 pos=3/289/290
     Readdir64 dir=0x805ad10 pos=4/289/290
     ...
     Readdir64 dir=0x805ad10 pos=287/289/290
     Readdir64 dir=0x805ad10 pos=288/289/290
     Readdir64 dir=0x805ad10 pos=289/289/290
     Readdir64 dir=0x805ad10 pos=0/289/290
     Readdir64: dirstruct->dp=(nil)
     Readdir64: ds=(nil)
     Segmentation fault (core dumped)

Always.  The "rm -r" loops over the directory, as show above,
and then tries to re-access entry 0 somehow, at which point
it discovers that it's been NULLed out.

Which is weird, because the local seekdir() was never called,
and the code never zeroed/freed that memory itself
(I've got printfs in there..).

Nulling out the qsort has no effect, and smaller/larger
ALLOC_STEPSIZE values don't seem to matter.

But.. when the entire tree is in RAM (freshly unpacked .tar),
it seems to have no problems with it.  As opposed to an uncached tree.

Peculiar.. I wonder where the bug is ?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/