linux-ext4 - Re: Wrong calculation of space remaining on a 32 bit system.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20081103202440.GG29102@mit.edu>
Date:	Mon, 3 Nov 2008 15:24:40 -0500
From:	Theodore Tso <tytso@....edu>
To:	François Valenduc 
	<francois.valenduc@...ablenet.be>
Cc:	Graham Murray <graham@...rray.org.uk>, linux-ext4@...r.kernel.org
Subject: Re: Wrong calculation of space remaining on a 32 bit system.

On Mon, Nov 03, 2008 at 07:28:34PM +0100, François Valenduc wrote:
> 
> As you may have guessed, I created a logical volume for the portage tree
>  of gentoo. It indeeds contains a lot of small files with all the
> ebuilds. I use XFS for the portage tree on the 64 bits computer. Maybe
> ext4 is not appropriate for this case. Maybe I would have the same
> problem also on the 64 bits computer.

There's nothing wrong with using ext4 for this case; it's just that
you do need to specify the appropriate average inode size.  An average
inode size of 16k works fine for most applications, but not for this
one.  You could calculate an appropriate inode ratio by figuring the
average size of the files used on your system, which I would guess is
somewhere between 4k and 8k, and then this wouldn't be a problem.

I don't know a whole lot about Gentoo's ebuilds, and why it creates so
many small files but if it is common, we could create an entry in
/etc/mke2fs.conf, i.e., like this:

[fs_types]
	portage = {
		inode_ratio = 8192
	}

Now if you run "mke2fs -t portage", it would automatically create a
filesystem with an inode ratio of 8k.  

As another example, suppose a spy agency needed to create filesystem
where they stored the results of "pen registers", which record the
time, date, source and destination of phone calls[1].  And suppose
they decide to store each intercept as a separate file, and each file
on average is less than 1k.  (No kidding, I was once at a "next
generation filesystem" requirements session at Sandia National Labs
some 9 years ago where they asked for something very similar to this
--- the ability to write huge numbers of itsy-bitsy files very
quickly, in contrast to the a-bomb simulation folks, who wanted the
ability to write a few really big files very quickly.)  Then they
might want to do something like this:

[fs_types]
	nsa = {
	    inode_ratio = 1024
	    blocksize = 1024
	}

(Alternatively, they might want to consider storing their intercepts
in something *other* than a filesystem, like say a database instead.  :-)

[1] http://www.capsteps.com/sounds/nsa-glory.mp3

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html