linux-ext4 - Re: [PATCH] ext4: add lazyinit stats support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20160517061323.GY7799@thunk.org>
Date:	Tue, 17 May 2016 02:13:23 -0400
From:	Theodore Ts'o <tytso@....edu>
To:	Shuichi Ihara <sihara@....com>
Cc:	Eric Sandeen <sandeen@...hat.com>,
	Wang Shilong <wangshilong1991@...il.com>,
	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>,
	"adilger@...ger.ca" <adilger@...ger.ca>
Subject: Re: [PATCH] ext4: add lazyinit stats support

On Tue, May 17, 2016 at 05:36:57AM +0000, Shuichi Ihara wrote:
> Sure, disabling layzinit is an option, but our single disk size is
> more than 60TB to make several petabyte system with Lustre.
> It takes time in a long while to be completion of mkfs without
> layzinit and we can't do anything until mkfs is done.  layzinit is
> still help, we can mount filesystem and do something (create Lustre
> and testing from client, etc) in parallel under lazyinit is running
> behind.

Ah, I'm used to doing single disk benchmarks where in the interests of
getting numbers which are as reproducible as possible, I always run
the performance benchmarks on a freshly initialized file system, and
so I don't want to do anything between when the file system is
initialized and when the workload is started --- or if I'm going to be
running a test on an aged file system, I'll have a standardized file
system aging tool with a fixed random seed, and/or a file system image
which I'll dd into place so that the starting point for the
performance evaluation is exactly the same for each benchmark run.

It didn't occur to me that you might want to be using the file system
while lazy init was taking place, and then later on, doing the
performance benchmarks on a used, randomly dirtied file system.  Hence
my assumption that the benmarking shell script[1] would not be doing
anything at all while it waited for lazy init to be finished.

[1] In the interests of keeping the results as consistent as possible,
I tend to have benchmarking scripts that do all of the workload setup
and test running and data collection done in an automated fashion, so
I can kick off a benchmarking run and come back the next day (or after
the weekend) with all of the results collected.

	    	       	    	  - Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html