linux-ext4 - e2fsck performance with uninit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20070410215040.GH5967@schatzie.adilger.int>
Date:	Tue, 10 Apr 2007 15:50:40 -0600
From:	Andreas Dilger <adilger@...sterfs.com>
To:	linux-ext4@...r.kernel.org
Cc:	"Andrew C. Uselton" <uselton2@...l.gov>
Subject: e2fsck performance with uninit_groups patch

I'm forwarding some interesting performance results for running with the
uninit_groups feature that I posted patches here recently.  The executive
summary is that fsck time with the patches is directly related to the
number of inodes in use, instead of being related to the number of inodes
in the whole filesystem.

In the test results this shaves about 1500s (almost 1/2) of the time off
e2fsck on an empty filesystem, and the majority of filesystems I looked
at show only 1-5% of the inodes in use.

Andrew Uselton <acu@...l.gov> wrote:
> The test proceeded by filling up the available inodes on a 2TB LUN
> (there were 244M inodes available), and testing the time to fsck the
> file system every additional 2% full.  We only used the new "sped up"
> fsck code, but emulated the older code by turning on and then turning
> off the feature at each 2% point.  Each check included 3 fsck runs: 
> - the  first to establish the "high water mark" data structures that are
>   the significant feature used by the sped up code.   This takes about as
>   long as the old fsck
> - the second measures fsck time when the feature is enabled and the high
>   water marks are in place
> - the third removes the high water mark data structures as the feature
>   is turned off.  This is also similar to running the old fsck.

> data for the test is included in the first attachment: fsck.data
> graph showing the old and new fsck performance numbers is: fsck-speedup.png.  
> Two additional graphs show the breakdown of the fsck times into passes
> 1 through 5:
> fsck-est-hw.png - time to fsck completion to establish the high water marks
> fsck-use-hw.png - time to fsck completion using the high water marks


# 1 - % of inodes in use
# 2 - fsck time to establish high water mark
# 3 - fsck time using high water mark
# 4 - fsck time to remove high water mark
# %	fsck-est-hw	fsck-use-hw	fsck-rem-hw
0	1340.02		22.54		1334.34
20	1867.73		600.77		1650.39
40	2193.05		1188.49		1979.05
60	2518.19		1782.43		2316.08
80	2849.34		2382.62		2653.07
98	3149.81		2929.43		2960.54


# fsck to establish high water mark
# %	Pass 1:		Pass 2: Pass 3: Pass 4:	Pass 5:
0	1113.08		0.39 	0.05 	6.66 	218.50
40	1640.15		107.02 	222.9 	10.11 	12.88 
80	2191.88		221.74 	223.0 	13.60 	13.16 
98	2438.98		275.41 	220.7 	15.07 	13.46 


# fsck using high water mark
# %	Pass 1:		Pass 2: Pass 3: Pass 4:	Pass 5:
0	2.52		0.35 	0.05 	6.64 	12.49 
40	974.02		109.89 	0.05 	10.09 	93.35 
80	1968.09		225.21 	0.07 	13.55 	174.57 
98	2420.09		278.45 	0.07 	15.12 	214.55 

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.


Download attachment "fsck-speedup.png" of type "image/png" (4994 bytes)

Download attachment "fsck-est-hw.png" of type "image/png" (5184 bytes)

Download attachment "fsck-use-hw.png" of type "image/png" (5467 bytes)

View attachment "fsck.data" of type "text/plain" (7728 bytes)