linux-ext4 - Re: how to scale root-reserved space going forward...

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-id: <20090302085632.GM3199@webber.adilger.int>
Date:	Mon, 02 Mar 2009 01:56:32 -0700
From:	Andreas Dilger <adilger@....com>
To:	Theodore Tso <tytso@....edu>
Cc:	Eric Sandeen <sandeen@...hat.com>,
	ext4 development <linux-ext4@...r.kernel.org>
Subject: Re: how to scale root-reserved space going forward...

On Mar 01, 2009  21:47 -0500, Theodore Ts'o wrote:
> This is a reasonable question.  What would be great is if we could get
> a benchmarking team to fill an ext4 filesystem with files.  The simple
> thing would be if we did something fixed --- say, 50 files per
> directory, each file 100k, and say 10 subdirectories in each
> directory, to some fixed depth, and with a filesystem size of at least
> 8 gigabytes (which would give us at least 16 flex groups with the
> default flex size of 16) --- and then filled each filesystem to from
> 0% to 90% in increments of 10%, and from 90% to 99% in increments of
> 1%, and then ran some throughput benchmark like bonnie on the mostly
> filled filesystem.

We've done tests like this, and it is important to take the inner vs.
outer cyliners into account.  It can happen that even a "perfectly"
allocated filesystem will appear to show slowdowns in performance as
it gets full, yet this is partitially due to physical disk layout issues.

> A better filler would probably use a random file sizes with a average
> size of say 64k, but with outliers from 4k to 128 megs, and a similar
> random distribution of number of files per directory, and number of
> subdirectories and depth of subdirectories.

You describe the Reiserfs "Mongo" benchmark.

> I suppose it would be good to do one set of charts with a filesystem
> size of 8 gigs, and another at 80 gigs and 800 gigs, and see if the
> shape of the filesystem curve changes at scale.  Once we have that, we
> would be in a position to make a reasonable set of defaults.
> 
> Or we could just guess and come up with some percentage figure that
> sounds good.  :-)

I suspect that at a certain filesystem size, there isn't much benefit
in having more reserved space.  If we keep 50GB of reserved space then
this is likely to contain a decent amount of 1MB free chunks, which is
what we really care about.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html