linux-ext4 - Re: dump ext4 performance degrades linearly as disk fills

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 16 Jun 2014 15:48:18 -0600
From:	Andreas Dilger <adilger@...ger.ca>
To:	"Joseph D. Wagner" <joe@...ephdwagner.info>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: dump ext4 performance degrades linearly as disk fills

On Jun 15, 2014, at 11:43 PM, Joseph D. Wagner <joe@...ephdwagner.info> wrote:
> Background:
> - I use lvm snapshots on ext4 for backup.  I use dump to backup the snapshots.  The backup goes to an external hard drive over usb 3.0. The external hard drive has 1 partition formatted with ext4.
> 
> Problem:
> - I've noticed the performance of dump to my ext4 backup degrades linearly as the disk fills.  In the output, the kbps in the status line keeps getting lower as the percent complete keeps getting higher.

One major cause for performance degradation as a disk fills is that the
inner tracks of a disk (higher block addresses) have about 1/2 the
throughput of the outer tracks (lower block addresses).  This means it
is basically impossible to get more than 50% of the performance from
blocks that are allocated on the higher addresses in the filesystem.

You could test this by creating a partition on the low blocks of the disk,
one that consumes most of the remaining space, and one at the high blocks
of the disk, create filesystems on the low- and high-block partitions, and
then running benchmarks on them.

Cheers, Andreas

> My Thoughts So Far:
> - I suspect that either 1) dump is doing something which lowers performance as the backup progresses, or 2) the ext4 algorithm for finding and allocating free blocks is vulnerable to performance degradation as the volume fills.
> 
> - I haven't tested this thoroughly.  However, performance appears to improve when I clear out the external drive and do a fresh, full dump (-0), and performance appears to remain degraded on incremental backups on a nearly full volume.  This leads me to suspect #2.
> 
> Help from you:
> - What steps can I take to isolate the cause of the problem?  If there's any information I can provide, please let me know.
> 
> What I've Done:
> - I've run both the short and long SMART tests on the external drive, which came back clean.  They also told me the relocated sector count was zero.
> 
> - I also made sure the disk was connecting in usb 3.0 mode.
> 
> - I changed the external drive ext4 mount from journal=ordered to journal=writeback.  The performance difference was nominal.
> 
> Any insight would be appreciated.
> 
> Thanks.
> 
> Joseph D. Wagner
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cheers, Andreas






Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)