linux-ext4 - Re: [PATCH] ext4: add lazyinit stats support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <71C04976-407D-4B88-9EFD-923636E69ACB@ddn.com>
Date:	Tue, 17 May 2016 05:36:57 +0000
From:	Shuichi Ihara <sihara@....com>
To:	Theodore Ts'o <tytso@....edu>, Eric Sandeen <sandeen@...hat.com>
CC:	Wang Shilong <wangshilong1991@...il.com>,
	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>,
	"adilger@...ger.ca" <adilger@...ger.ca>
Subject: Re: [PATCH] ext4: add lazyinit stats support

Sure, disabling layzinit is an option, but our single disk size is more than 60TB to make several petabyte system with Lustre.
It takes time in a long while to be completion of mkfs without layzinit and we can't do anything until mkfs is done.
layzinit is still help, we can mount filesystem and do something (create Lustre and testing from client, etc) in parallel under lazyinit is running behind.


So, we still want to see how lazyinit is going and know estimation of finish it to do really performance intensive testing after lazyinit finished.

Thanks
Ihara 

On 5/17/16, 1:45 PM, "Theodore Ts'o" <tytso@....edu> wrote:

>On Mon, May 16, 2016 at 11:22:35PM -0500, Eric Sandeen wrote:
>> 
>> Sure thing; I'm still on the fence about usefulness, because if
>> anyone really cares to wait for it to hit zero, they probably
>> should have just changed their mkfs options to disable lazyinit.
>
>Indeed, if you're going to stall your test startup until lazyinit is
>done, it will be faster to let mke2fs initialize the inode table
>blocks, since lazyinit deliberately rate limits how much I/O it sends
>down to the disk in order to avoid impacting user latencies.  So i'd
>recommend "mke2fs -E lazy_itable_init=0,lazy_journal_init=0" instead
>of patching the kernel and then waiting for lazy itable init to
>complete.  I think you'll be happier with how long this takes.
>
>BTW, if you care about 99.9 percentile (long tail) latency as a
>performance metric, it's a good idea to disable lazy inode table init
>and just let mke2fs take its own sweet time.  This is what we do
>inside Google, since we care very much about long tail latencies on a
>number of our workloads.
>
>Cheers,
>
>						- Ted