linux-ext4 - Re: Strange (varying) limit on number files in directory ?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <7A464EA2-9841-4C3D-B7D1-3D0711951F5E@whamcloud.com>
Date:	Fri, 13 May 2011 08:29:32 -0600
From:	Andreas Dilger <adilger@...mcloud.com>
To:	Mateusz Korniak <mateusz-lists@....gliwice.pl>
Cc:	"linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: Re: Strange (varying) limit on number files in directory ?

On 2011-05-13, at 5:36 AM, Mateusz Korniak <mateusz-lists@....gliwice.pl> wrote:
> I am hitting strange number of files inside directory limitation.
> I am able to create only around 120k files.
> Number varies [1] as seems I am able to add files with some filenames but with 
> other creation fails [2].
> 
> Files have structure <small_ind><sha/md5sum> [3].
> All on linux box [4],[5], having plenty of frees space [6].
> Filesystem fscked  - everything ok.
> 
> 
> [3]:
> 494_0cf1500e1ecbeecb3282b1e68b2df53b77d06665  
> 494_40654eec2dd15e4065e49faa2ea226dd25191457  
> 
> [5]:
> tune2fs -l /dev/md7
> tune2fs 1.41.11 (14-Mar-2010)
> 
> Block count:              117190656
> 
> Block size:               1024

Limit is because you must have formatted the filesystem with 1023-byte blocksize. There is a 2-level limit on htree directories, and with the long filenames you are using you only get about 16 entries in each leaf block. With 1024-byte blocks there can be at most (1024-16)/8*(1024-8)/8*3/4= 192k entries with perfect hashing.

In order to exceed this limit you unfortunately need to format your fs with a larger blocksize (or fix htree to allow more than depth 2).

Andreas
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html