linux-ext4 - Re: dump ext4 performance degrades linearly as disk fills

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Tue, 29 Jul 2014 15:48:27 -0700
From:	"Joseph D. Wagner" <joe@...ephdwagner.info>
To:	Phillip Susi <psusi@...ntu.com>
Cc:	linux-ext4@...r.kernel.org, Phillip Susi <phillsusi@...il.com>
Subject: Re: dump ext4 performance degrades linearly as disk fills

On 07/29/2014 1:55 pm, Phillip Susi wrote:

> Just thought I'd suggest two things you might try:
> 
> 1) dump to /dev/null to make sure it is a read problem, and not a
> problem writing the output file.

It's not a read problem.  It's a write problem.  I confirmed this by 
reformatting the *destination* media from ext4 to xfs.  The problem 
disappears on xfs.  The problem also disappears on ext4 without a 
journal, suggesting the problem is in the journal.

Curiously, formatting with this option also makes the problem disappear:
-E lazy_itable_init=0,lazy_journal_init=0

I filed all of this in a kernel bug report.  You may want to take a 
look.
https://bugzilla.kernel.org/show_bug.cgi?id=78651

> 2) You might try throwing e2defrag at the fs and see if that helps (
> after a full backup of course ). In the past I have seen significant
> slowness when dumping due to my Maildir being fragmented all to hell.
> If you have any sets of large numbers of relatively small files, that
> could be what you are seeing. An e2defrag pass will completely clear
> this up.

Since the problem disappears when using different options on the 
*destination* media, I'm going to assume this is unnecessary for the 
source files.  As for the img file dump creates, all the extents are 128 
MB in size (the maximum extent size on ext4) except for the last one.

Joseph D. Wagner
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html