linux-ext4 - Re: ext3_dx_add_entry complains about Directory index full

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150204135215.GA15159@aepfle.de>
Date:	Wed, 4 Feb 2015 14:52:16 +0100
From:	Olaf Hering <olaf@...fle.de>
To:	Andreas Dilger <adilger@...ger.ca>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: ext3_dx_add_entry complains about Directory index full

On Wed, Feb 04, Andreas Dilger wrote:

> How many files/subdirs in this directory?  The old ext3 limit was 32000
> subdirs, which the dir_index fixed, but the new limit is 65000 subdirs
> without "dir_index" enabled.

See below:

> > # for t in d f l ; do echo "type $t: `find /media/BACKUP_OLH_500G/ -xdev -type $t | wc -l`" ; done
> > type d: 1051396
> > type f: 20824894
> > type l: 6876

> The 65000 subdir limit can be exceeded by turning on the "dir_nlink"
> feature of the filesystem with "tune2fs -O dir_nlink", to allow an
> "unlimited" number of subdirs (subject to other directory limits, about
> 10-12M entries for 16-char filenames).

I enabled this using another box, which turned the thing into an ext4
filesystem. Now ext4_dx_add_entry complains.

> The other potential problem is if you create and delete a large number
> of files from this directory, then the hash tables can become full and
> the leaf blocks are imbalanced and some become full even when many others
> are not (htree only has an average leaf fullness of 3/4 of each block).
> This could probably happen if you have more than 5M files in a long-lived
> directory in your backup fs.  This can be fixed (for some time at least)
> via "e2fsck -fD" on the unmounted filesystem to compact the directories.

Ok, will try that. Thanks.

Olaf
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html