linux-ext4 - ext2/3 create large filesystem takes too much time; solutions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <401f4f10609120407j6816372mfdfea392dcae9e00@mail.gmail.com>
Date:	Tue, 12 Sep 2006 14:07:34 +0300
From:	"Pavel Mironchik" <tibor0@...il.com>
To:	linux-ext4@...r.kernel.org
Subject: ext2/3 create large filesystem takes too much time; solutions

Hi,

Ext2/3 does erase of inode tables, when do creation of new systems.
This is very very long operation when the target file system volume is more than
2Tb. Other filesystem are not affected by such huge delay on creation of
filesystem. My concern was to improve design of ext3 to decrease time
consuption of creation large ext3 volumes on storage servers.
In general to solve problem, we should defer job of cleaning nodes to
kernel. In e2fsprogs there is LAZY_BG options but it  just avoids doing
erase of inodes only.

I see several solutions for that problem:
1) Add special bitmaps into fs header (inode groups descriptors?).
By looking at those bitmaps kernel could determine if inode is not cleaned, and
that inode will be propertly initialized.
2) Add special identifiers into inodes. If super block id != inode id
-> inode is dirty
and should be cleaned in kernel, where super block id is generated on
creation stage.

I choosed second (much easier - just few lines of code)
and implemented patch for e2fsprogs, kernel ext3. It is just proof of a concept.
With the help of this patch I could create  terrabytes volumes fast.
Of cource this patch will broke compatibility for existing filesystem.
More correctly is to choose first way and do not broke compatibility.

Writing this mail, I just want check if there is any interest for this problem
from community.
I would like to see that future ext4 filesystem will be created fast.
and I would be appreciate for  thoughts, remarks.
---------------------------
Pavel S. Mironchik

View attachment "ext3-kernel.patch" of type "text/x-patch" (3426 bytes)

View attachment "mkfs_and_fsck.patch" of type "text/x-patch" (4632 bytes)