linux-ext4 - dump ext4 performance degrades linearly as disk fills

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Sun, 15 Jun 2014 22:43:29 -0700
From:	"Joseph D. Wagner" <joe@...ephdwagner.info>
To:	linux-ext4@...r.kernel.org
Subject: dump ext4 performance degrades linearly as disk fills

Background:
- I use lvm snapshots on ext4 for backup.  I use dump to backup the 
snapshots.  The backup goes to an external hard drive over usb 3.0. The 
external hard drive has 1 partition formatted with ext4.

Problem:
- I've noticed the performance of dump to my ext4 backup degrades 
linearly as the disk fills.  In the output, the kbps in the status line 
keeps getting lower as the percent complete keeps getting higher.

My Thoughts So Far:
- I suspect that either 1) dump is doing something which lowers 
performance as the backup progresses, or 2) the ext4 algorithm for 
finding and allocating free blocks is vulnerable to performance 
degradation as the volume fills.

- I haven't tested this thoroughly.  However, performance appears to 
improve when I clear out the external drive and do a fresh, full dump 
(-0), and performance appears to remain degraded on incremental backups 
on a nearly full volume.  This leads me to suspect #2.

Help from you:
- What steps can I take to isolate the cause of the problem?  If there's 
any information I can provide, please let me know.

What I've Done:
- I've run both the short and long SMART tests on the external drive, 
which came back clean.  They also told me the relocated sector count was 
zero.

- I also made sure the disk was connecting in usb 3.0 mode.

- I changed the external drive ext4 mount from journal=ordered to 
journal=writeback.  The performance difference was nominal.

Any insight would be appreciated.

Thanks.

Joseph D. Wagner
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html