Date:	Mon, 03 Nov 2008 22:23:32 +0100
From:	François Valenduc 
	<francois.valenduc@...ablenet.be>
To:	Theodore Tso <tytso@....edu>
CC:	Graham Murray <graham@...rray.org.uk>, linux-ext4@...r.kernel.org
Subject: Re: Wrong calculation of space remaining on a 32 bit system.

Theodore Tso a écrit :
> On Mon, Nov 03, 2008 at 07:28:34PM +0100, François Valenduc wrote:
>   
>> As you may have guessed, I created a logical volume for the portage tree
>>  of gentoo. It indeeds contains a lot of small files with all the
>> ebuilds. I use XFS for the portage tree on the 64 bits computer. Maybe
>> ext4 is not appropriate for this case. Maybe I would have the same
>> problem also on the 64 bits computer.
>>     
>
> There's nothing wrong with using ext4 for this case; it's just that
> you do need to specify the appropriate average inode size.  An average
> inode size of 16k works fine for most applications, but not for this
> one.  You could calculate an appropriate inode ratio by figuring the
> average size of the files used on your system, which I would guess is
> somewhere between 4k and 8k, and then this wouldn't be a problem.
>
> I don't know a whole lot about Gentoo's ebuilds, and why it creates so
> many small files but if it is common, we could create an entry in
> /etc/mke2fs.conf, i.e., like this:
>
> [fs_types]
> 	portage = {
> 		inode_ratio = 8192
> 	}
>
> Now if you run "mke2fs -t portage", it would automatically create a
> filesystem with an inode ratio of 8k.  
>
> As another example, suppose a spy agency needed to create filesystem
> where they stored the results of "pen registers", which record the
> time, date, source and destination of phone calls[1].  And suppose
> they decide to store each intercept as a separate file, and each file
> on average is less than 1k.  (No kidding, I was once at a "next
> generation filesystem" requirements session at Sandia National Labs
> some 9 years ago where they asked for something very similar to this
> --- the ability to write huge numbers of itsy-bitsy files very
> quickly, in contrast to the a-bomb simulation folks, who wanted the
> ability to write a few really big files very quickly.)  Then they
> might want to do something like this:
>
> [fs_types]
> 	nsa = {
> 	    inode_ratio = 1024
> 	    blocksize = 1024
> 	}
>
> (Alternatively, they might want to consider storing their intercepts
> in something *other* than a filesystem, like say a database instead.  :-)
>
> [1] http://www.capsteps.com/sounds/nsa-glory.mp3
>
> 						- Ted
>
>   
Thanks for all your explanations. I finally found that 4096 seems to be
the more suitable ratio for the gentoo portage tree. Decreasing the
inode ratio to 8192 is not enough to avoid the problem. In fact the tree
consists of all the ebuilds which are all small text files located in a
lot of subdirectories. I have no idea how to compute the exact average
size but the most common size seems to be between 4 and 6 k. Then there
are all the source files which by default are located in the 
/usr/portage/distfiles directory. These can be very large (around 300 Mb
for openoffice).
So, as you suggested I add the following section in mke2fs.conf:

   portage = {
                features =
has_journal,extents,flex_bg,uninit_bg,dir_nlink,extra_isize
                inode_ratio=4096
        }

So running mke2fs -T portage creates indeeed an ext4 filesystem with the
appropriate inode ratio for the gentoo portage tree, without huge_file
support which is certainly not needed for this case.

François
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists