Message-ID: <53A04BEB.7000401@josephdwagner.info>
Date:	Tue, 17 Jun 2014 07:08:43 -0700
From:	"Joseph D. Wagner" <joe@...ephdwagner.info>
To:	"Theodore Ts'o" <tytso@....edu>
CC:	linux-ext4@...r.kernel.org
Subject: Re: dump ext4 performance degrades linearly as disk fills


On 06/16/2014 05:42 AM, Theodore Ts'o wrote:
> Something that might help is simply running "dumpe2fs /dev/sdXX > 
> /dev/null" or "e2freefrag /dev/hdXX > /dev/null" after you mount the 
> file system and before you kick off the backup. This will load all of 
> the block allocation bitmaps into the buffer cache, and the libext2fs 
> functions used by dumpe2fs and e2freefrag will do so much more 
> efficiently than the kernel code will as it demand-loads the bitmap 
> blocks. Hope this helps! - Ted 
I just got done trying that.  Below is what I am seeing.  It seemed to 
do well up until 20%, but then performance drops.  Between 31% and 33%, 
I reran dumpe2fs in case this was some sort of cache poison issue.  As 
you can see, it didn't improve performance at the time.

FYI, lvbackup is a script I wrote to 1) create to lvm snapshot, 2) run 
dump, and 3) drop the snapshot.

# su -c 'cd /root; dumpe2fs /dev/mapper/backup > /dev/null; nice ionice 
-c3 /usr/local/bin/lvbackup /dev/data/home /mnt/backup'
Password:
dumpe2fs 1.42.8 (20-Jun-2013)
   DUMP: mount: bad UUID
   DUMP: Date of this level 0 dump: Mon Jun 16 20:08:57 2014
   DUMP: Dumping /dev/data/backup-data-home (an unlisted file system) to 
/mnt/backup/backup-data-home-20140616.200856-0.img
   DUMP: Label: none
   DUMP: Writing 64 Kilobyte records
   DUMP: mapping (Pass I) [regular files]
   DUMP: mapping (Pass II) [directories]
   DUMP: estimated 1722505007 blocks.
   DUMP: writing QFA positions to 
/mnt/backup/backup-data-home-20140616.200856-0.qfa
   DUMP: Volume 1 started with block 1 at: Mon Jun 16 20:09:11 2014
   DUMP: dumping (Pass III) [directories]
   DUMP: dumping (Pass IV) [regular files]
   DUMP: 1.92% done at 110516 kB/s, finished in 4:14
   DUMP: 4.05% done at 116142 kB/s, finished in 3:57
   DUMP: 6.27% done at 119939 kB/s, finished in 3:44
   DUMP: 8.51% done at 122095 kB/s, finished in 3:35
   DUMP: 10.73% done at 123209 kB/s, finished in 3:28
   DUMP: 12.89% done at 123382 kB/s, finished in 3:22
   DUMP: 14.98% done at 122868 kB/s, finished in 3:18
   DUMP: 17.05% done at 122380 kB/s, finished in 3:14
   DUMP: 18.98% done at 121068 kB/s, finished in 3:12
   DUMP: 20.37% done at 116976 kB/s, finished in 3:15
   DUMP: 21.72% done at 113360 kB/s, finished in 3:18
   DUMP: 23.12% done at 110641 kB/s, finished in 3:19
   DUMP: 24.40% done at 107777 kB/s, finished in 3:21
   DUMP: 25.76% done at 105628 kB/s, finished in 3:21
   DUMP: 27.08% done at 103641 kB/s, finished in 3:21
   DUMP: 28.30% done at 101527 kB/s, finished in 3:22
   DUMP: 29.50% done at 99508 kB/s, finished in 3:23
   DUMP: 30.79% done at 98101 kB/s, finished in 3:22
   DUMP: 31.88% done at 96210 kB/s, finished in 3:23
   DUMP: 33.31% done at 95509 kB/s, finished in 3:20
   DUMP: 34.50% done at 94219 kB/s, finished in 3:19
   DUMP: 35.86% done at 93449 kB/s, finished in 3:17
   DUMP: 37.25% done at 92860 kB/s, finished in 3:14
   DUMP: 38.64% done at 92323 kB/s, finished in 3:10
   DUMP: 39.91% done at 91559 kB/s, finished in 3:08
   DUMP: 41.18% done at 90845 kB/s, finished in 3:05
   DUMP: 42.59% done at 90476 kB/s, finished in 3:02
   DUMP: 43.88% done at 89885 kB/s, finished in 2:59
   DUMP: 45.09% done at 89188 kB/s, finished in 2:56
   DUMP: 46.45% done at 88794 kB/s, finished in 2:53
   DUMP: 47.77% done at 88370 kB/s, finished in 2:49
   DUMP: 49.12% done at 88029 kB/s, finished in 2:45
   DUMP: 50.36% done at 87518 kB/s, finished in 2:42
   DUMP: 51.53% done at 86929 kB/s, finished in 2:40
   DUMP: 52.65% done at 86276 kB/s, finished in 2:37
   DUMP: 53.97% done at 85979 kB/s, finished in 2:33
   DUMP: 55.30% done at 85720 kB/s, finished in 2:29
   DUMP: 56.58% done at 85388 kB/s, finished in 2:25
   DUMP: 57.91% done at 85158 kB/s, finished in 2:21
   DUMP: 59.30% done at 85024 kB/s, finished in 2:17
   DUMP: 60.66% done at 84848 kB/s, finished in 2:13
   DUMP: 61.94% done at 84576 kB/s, finished in 2:09
   DUMP: 63.21% done at 84299 kB/s, finished in 2:05
   DUMP: 64.24% done at 83730 kB/s, finished in 2:02
   DUMP: 65.37% done at 83309 kB/s, finished in 1:59
   DUMP: 66.48% done at 82892 kB/s, finished in 1:56
   DUMP: 67.69% done at 82599 kB/s, finished in 1:52
   DUMP: 68.76% done at 82161 kB/s, finished in 1:49
   DUMP: 69.90% done at 81788 kB/s, finished in 1:45
   DUMP: 70.95% done at 81365 kB/s, finished in 1:42
   DUMP: 72.13% done at 81094 kB/s, finished in 1:38
   DUMP: 73.33% done at 80861 kB/s, finished in 1:34
   DUMP: 74.42% done at 80510 kB/s, finished in 1:31
   DUMP: 75.51% done at 80186 kB/s, finished in 1:27
   DUMP: 76.65% done at 79919 kB/s, finished in 1:23
   DUMP: 77.72% done at 79577 kB/s, finished in 1:20
   DUMP: 78.74% done at 79212 kB/s, finished in 1:17
   DUMP: 79.92% done at 79016 kB/s, finished in 1:12
   DUMP: 81.00% done at 78728 kB/s, finished in 1:09
   DUMP: 82.20% done at 78564 kB/s, finished in 1:05
   DUMP: 83.33% done at 78344 kB/s, finished in 1:01
   DUMP: 84.39% done at 78048 kB/s, finished in 0:57
   DUMP: 85.19% done at 77542 kB/s, finished in 0:54
   DUMP: 86.20% done at 77237 kB/s, finished in 0:51
   DUMP: 87.19% done at 76919 kB/s, finished in 0:47
   DUMP: 88.13% done at 76568 kB/s, finished in 0:44
   DUMP: 89.14% done at 76289 kB/s, finished in 0:40
   DUMP: 90.08% done at 75961 kB/s, finished in 0:37
   DUMP: 91.20% done at 75791 kB/s, finished in 0:33
   DUMP: 92.34% done at 75646 kB/s, finished in 0:29
   DUMP: 93.56% done at 75563 kB/s, finished in 0:24
   DUMP: 94.69% done at 75415 kB/s, finished in 0:20
   DUMP: 95.77% done at 75203 kB/s, finished in 0:16
   DUMP: 96.83% done at 74995 kB/s, finished in 0:12
   DUMP: 97.79% done at 74733 kB/s, finished in 0:08
   DUMP: 98.74% done at 74465 kB/s, finished in 0:04
   DUMP: 99.67% done at 74171 kB/s, finished in 0:01
   DUMP: Closing /mnt/backup/backup-data-home-20140616.200856-0.img
   DUMP: Volume 1 completed at: Tue Jun 17 02:36:40 2014
   DUMP: Volume 1 1722545856 blocks (1682173.69MB)
   DUMP: Volume 1 took 6:27:29
   DUMP: Volume 1 transfer rate: 74091 kB/s
   DUMP: 1722545856 blocks (1682173.69MB) on 1 volume(s)
   DUMP: finished in 23249 seconds, throughput 74091 kBytes/sec
   DUMP: Date of this level 0 dump: Mon Jun 16 20:08:57 2014
   DUMP: Date this dump completed:  Tue Jun 17 02:36:40 2014
   DUMP: Average transfer rate: 74091 kB/s
   DUMP: Archiving dump to 
/mnt/backup/backup-data-home-20140616.200856-0.toc
   DUMP: DUMP IS DONE
   Logical volume "backup-data-home" successfully removed

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists