linux-ext4 - Re: mke2fs with bigalloc is too slow

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAA0+VHx5=hYPOefnA-5eTrwFMtsg4sLujCLB91fGx2eoZrP0+w@mail.gmail.com>
Date:	Fri, 15 Mar 2013 01:53:58 +0400
From:	Andrey Sidorov <qrxd43@...orola.com>
To:	"Theodore Ts'o" <tytso@....edu>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: mke2fs with bigalloc is too slow

On Wed, Mar 13, 2013 at 11:00 PM, Theodore Ts'o <tytso@....edu> wrote:
> Actually, these days we use a rbtree to encode the bitmap.  That means
> that it should be possible to create a very efficient find_first_set
> and find_first_zero functions.  This would work especially well for
> mke2fs since the allocation bitmap will be mostly empty.  This I think
> would improve things dramatically.

Oh, thanks for pointing that! I was mostly reading v1.42 that is more
than year old and missed the fact 1.42.7 has rbtree bitmap
implementation.
That explains why formatting bigalloc became slower compared to 1.42.
In this case ffs/ffz seem like a right choice, I'll try them. I'll
also try mirroring as it won't waste much RAM and might be faster.

> Fragmenting the block allocation bitmaps slows down operations that
> need to read in multiple block bitmaps in sequence.  This includes
> dumpe2fs, e2fsck, and in some cases, block allocation.  So making this
> change is not zero-cost.  I agree that 30 seconds for mke2fs is
> non-optimal, but I'm surprised you're finding this to be a
> showstopper.  I assume you're worried about substantially bigger file
> systems than just 1TB?

1TB is what we have. It is a show stopper because it increases first
boot time a lot compared to total boot time. This is bad because first
boot happens on factory and thus should be as quick as possible.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html