linux-ext4 - Re: Ininitial e2fsprogs TODO list (please expand)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-id: <20080416033002.GZ3106@webber.adilger.int>
Date:	Tue, 15 Apr 2008 21:30:02 -0600
From:	Andreas Dilger <adilger@....com>
To:	"Jose R. Santos" <jrs@...ibm.com>
Cc:	linux-ext4 <linux-ext4@...r.kernel.org>
Subject: Re: Ininitial e2fsprogs TODO list (please expand)

On Apr 15, 2008  11:52 -0500, Jose R. Santos wrote:
> As discuss on the call yesterday, some folks (my self included) really
> want a TODO list to help them keep track of what things are left undone
> in e2fsprogs as we try to get ext4 out the door.  Here is my initial
> list of items that still need addressing.  Hopefully we can expand this
> list and document it somewhere like the ext4 wiki or the SourceForge
> bug tracker.
> 
> - Rename uninit_groups to uninit_bg to be consistent with other
> defined features.  Retain the old name for historical purpose.
> 
> - The return value of ext2fs_super_and_bgd_loc() is not to be trusted.
> Document this in the source code.
> 
> - Make sure ext2fs_super_and_bgd_loc() does not get used anywhere where
> the return value is expected to be accurate (aside from mke2fs).
> 
> - Remove lazy_bg feature from being set in mke2fs.  Feature has been
> declare a dangerous hack by its creator, remove it to avoid people
> building on top of it.
> 
> - Add flex_bg meta-data grouping support.
> 
> - Remove support for not zeroing the inode tables from the
> uninit_groups patches.  This support is dangerous without a proper
> kernel thread that zeros them in the background when the filesystem is
> mounted.  Depends on the lazy_bg removal.

Something was lost in translation here.  The uninit_groups feature DOES
zero the inode tables by default, and marks the groups with ITABLE_ZEROED.
It is only if "-O uninit_groups,lazy_bg" are both given at the same time
that the itable is not initialized.  That is no different than if lazy_bg
was given by itself.

So nothing needs to be done in e2fsprogs until some time after the kernel
is updated to do the zeroing.

> - Activate undo-manager in mke2fs only when inode tables are not being
> zeroed.  Undo-manager is horribly slow if we need to store the
> information of all the blocks that have been zeroed during mke2fs.  The
> amount of storage needed for the undo on a 16TB filesystem could be
> problematic.  Depends on kernel thread inode table zeroing.
> 
> - Make a 64-bit clean API that extends the existing one.  The current
> API can not support larger than 32-bit blocks so a new set API calls is
> need in order to provide large filesystem support and retain backwards
> compatibility with the old API.
> 
> - 64-bit bitmap interface.  In order to support larger than 32-bit
> blocks, a new bitmap interface is needed that can retain ABI
> compatibility with the old one.

There are some notes on implementing more efficient bitmaps in
https://bugzilla.lustre.org/show_bug.cgi?id=12202

Even without 64-bit filesystems the memory consumption of e2fsck
can be quite high (2^32 blocks ~= 2^32 bytes of RAM for e2fsck).

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html