linux-ext4 - Re: [PATCH] mk2fs lazy_journal

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-id: <7095A240-FB57-4C33-8EE8-33D88B500319@sun.com>
Date:	Tue, 16 Feb 2010 15:15:19 -0700
From:	Andreas Dilger <adilger@....com>
To:	"Theodore Ts'o" <tytso@....edu>
Cc:	ext4 development <linux-ext4@...r.kernel.org>,
	Shuichi Ihara <ihara@....com>
Subject: Re: [PATCH] mk2fs lazy_journal_init option

On 2010-02-10, at 04:44, Andreas Dilger wrote:
> Attached is a patch to skip zeroing of the journal if the
> "-E lazy_journal_init" option is given to mke2fs (named to
> complement the "-E lazy_itable_init" option).  This can
> speed up format time significantly for large journal devices.
> There's only a short-term risk of problems with uninitialized
> journal, until the journal has been overwritten once.
>
> Patch has been lightly tested, showing mke2fs times steady
> at 14s for a 40GB filesystem, regardless of journal size,
> while previously it took up to 45s for an internal 2GB journal.

While testing this patch more thoroughly, we uncovered a bug
in the mke2fs/libext2fs code.  It seems that when running:

mke2fs -J size=X -O extents /dev/XXX

for any size > 512 the journal creation time is growing
exponentially:

no journal-> 12s
size=128 ->  14s
size=256 ->  16s
size=512 ->  21s
size=768 -> 143s
size=1024-> 298s
size=1280-> 663s

We wanted originally to use size=4000, but this took so
long we thought it was hung, and started investigating.

This happens even without the "-E lazy_itable_init" option.
Running ltrace on mke2fs shows lots of zero writes (to be
expected for journal zeroing) followed by a single read
(completes quickly) and many thousands of memcpy() calls.
The mke2fs program is completely CPU bound (99.9% user).

Running with the "-E lazy_itable_init" the writes/reads go
away, and all that is left is an endless stream of memcpy().

It seems to loop in ext2fs_block_iterate2->mkjournal_proc()
forever:

426            for (blockcnt = extent.e_lblk, j = 0;
427                 j < extent.e_len;
428                 blk++, blockcnt++, j++) {
429                    new_blk = blk;
430                    r = (*ctx.func)(fs, &new_blk, blockcnt,
431                                    0, 0, priv_data);


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html