linux-ext4 - [PATCH RFC] e2fsprogs: Speed up ext2fs_new

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <54E7C642.3020003@linux.intel.com>
Date:	Fri, 20 Feb 2015 15:41:54 -0800
From:	Darren Hart <dvhart@...ux.intel.com>
To:	linux-ext4@...r.kernel.org
CC:	tytso@....edu, darrick.wong@...cle.com,
	richard.purdie@...uxfoundation.org, liezhi.yang@...driver.com
Subject: [PATCH RFC] e2fsprogs: Speed up ext2fs_new_block2

Ted and Darrick,

We're back with a Yocto Project use case. We use mkfs.ext4 to create our
root filesystems. For one of our larger images including an X desktop
and an SDK (toolchain, etc.), it took over 8 minutes to complete.
Richard Purdie noticed most of this time was in ext2fs_new_block2. He
provided this small hack to test an idea and it has reduced the runtime
to 35 seconds. The images function as expected.

Can you have a look and see if there some value in this approach, or if
perhaps it might lead to a more appropriate/correct solution?

Thanks!

Darren

(What follows is the test patch from Richard for the Yocto Project)

The comment to this function says:

"""
  * Stupid algorithm --- we now just search forward starting from the
  * goal.  Should put in a smarter one someday....
"""

This adds in a rather hacky algorthim which starts where we finished
searching previously using a static variable rather than starting
from scratch if a hint isn't provided.

This was after noticing that mkfs.ext4 -F X -d Y was spending *lots*
of time in ext2fs_new_block2 called from ext2fs_bmap from
ext2fs_file_write().

Numbers wise, this took a core-image-sato-sdk mkfs time from over
8 minutes to around 35 seconds.

Upstream-Status: Pending

RP 2015/02/20

Index: e2fsprogs-1.42.9/lib/ext2fs/alloc.c
===================================================================
--- e2fsprogs-1.42.9.orig/lib/ext2fs/alloc.c
+++ e2fsprogs-1.42.9/lib/ext2fs/alloc.c
@@ -160,6 +160,8 @@ errcode_t ext2fs_new_inode(ext2_filsys f
 	return 0;
 }

+static blk64_t last_goal = 0;
+
 /*
  * Stupid algorithm --- we now just search forward starting from the
  * goal.  Should put in a smarter one someday....
@@ -170,6 +172,9 @@ errcode_t ext2fs_new_block2(ext2_filsys
 	blk64_t	i;
 	int	c_ratio;

+        if (!goal)
+            goal = last_goal;
+
 	EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);

 	if (!map)
@@ -194,6 +199,7 @@ errcode_t ext2fs_new_block2(ext2_filsys

 		if (!ext2fs_fast_test_block_bitmap2(map, i)) {
 			*ret = i;
+			last_goal = i;
 			return 0;
 		}
 		i = (i + c_ratio) & ~(c_ratio - 1);

-- 
Darren Hart
Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html