linux-ext4 - Filesystem pathname distributions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [day] [month] [year] [list]

Message-Id: <9DF36917-829A-4E19-939F-B8377148758B@whamcloud.com>
Date:	Tue, 8 May 2012 09:44:12 -0600
From:	Andreas Dilger <adilger@...mcloud.com>
To:	Zheng Liu <gnehzuil.liu@...il.com>
Cc:	Ext4 Developers List <linux-ext4@...r.kernel.org>
Subject: Filesystem pathname distributions

There is an interesting analysis of filesystem pathname distributions
available in a master's thesis at:

http://www.pdl.cmu.edu/PDL-FTP/HECStorage/Yifan_Final.pdf

This shows (at least for the filesystems analyzed) that median
directory sizes continue to be quite small.  Looking at figure
10, 60-90% of all directories have 8 or fewer entries in them,
and in many cases 50% of directories have only 1 or 2 entries.

It would be useful for you to run the fsstats tool (available
from http://www.pdsi-scidac.org/fsstats/) against some filesystems
that you have access to (e.g. typical distro desktop and home
directory, file servers, etc) and compute what fraction of the
files and directories could be stored inside the inode.

Similarly, and perhaps more importantly, for any filesystems you
have using the bigalloc feature, compare the actual size of
files and directories (only the filename + 10 bytes) to see how
many you could fit into an inode of a given size.  A default
256-byte inode has about 100 bytes of space, a 512-byte inode
has about 350 bytes of space, etc).  It would be useful to see
if there is a clear win for having larger inodes to hold small
files/directories in bigalloc filesystems, or whether this would
waste more space in total than is wasted by allocating a full
bigalloc block for each inode.

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html