linux-ext4 - Re: fsck performance.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <AANLkTikJs8mBHGvHkJAp-aC719aG6QSJaUH-uvwpx0JK@mail.gmail.com>
Date:	Mon, 21 Feb 2011 12:31:20 +0200
From:	Amir Goldstein <amir73il@...il.com>
To:	"Ted Ts'o" <tytso@....edu>
Cc:	Rogier Wolff <R.E.Wolff@...wizard.nl>, linux-ext4@...r.kernel.org
Subject: Re: fsck performance.

On Mon, Feb 21, 2011 at 1:41 AM, Ted Ts'o <tytso@....edu> wrote:
> On Mon, Feb 21, 2011 at 12:15:14AM +0100, Rogier Wolff wrote:
>> > BTW, my backup plan was to replace tdb with something else.  One of
>> > the candidates I was looking at was sqlite, but rumors of its speed
>> > deficiencies are making me worry that it won't be a good fit.  I don't
>> > want to use berk_db because it has a habit of changing API's
>> > regularly, and you can never be sure which version of berk_db
>> > different distributions might be using.  One package which I thought
>> > held promise was Koyoto Cabinet, but unfortunately, it's released
>> > under GPLv3, which makes it incompatible with the license used by
>> > e2fsprogs (which has to be GPLv2, since there are a few files which
>> > are shared with the Linux kernel).
>>

What about Tokyo Cabinet? it seems to be released under GPLv2.1.
Isn't it sufficient (or even better suiting) for scratch files?

>> Hmm. I'll take a look.
>
> If you could put a bit of time into this, that would be really great.
> I have a lot of things that I need to do at the moment, and trying to
> improve [scratch_files] performance is something I've known about for
> a while, but I just haven't had time to get to it.
>
> The fact that the problem can be solved by using 64-bit capable CPU's
> and a large swap space has kept this from rising to the top of the
> priority heap, but it is an important use case, since we do have NAS
> boxes that use cheap-sh*t 32-bit processors, and I'd like to be able
> to support them.  But I just don't have the time ATM, so if I can
> delegate this out to someone else, that would be really helpful.
>

CTERA uses *affordable* 32-bit processors for it's low-end NAS products,
which still need to be able to complete fsck of a 8TB fs in a reasonable time
(less than the time is takes the customer to loose it's patient and look for
alternative products).

We would be willing to invest resources for this cause,
should we know there is a promising path to follow.

One thing I am not sure I understand is (excuse my ignorance) why is the
swap space solution good only for 64bit processors?
Is it a common knowledge that fsck can require more than 3GB of memory?
If it is common knowledge, do you know of an upper limit (depending on fs size,
no. of inodes, etc)?

Thanks,
Amir.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html