lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Date:   Fri, 12 Nov 2021 19:52:37 +0000 (GMT)
From:   Mark Hills <>
Subject: Maildir quickly hitting max htree

Surprised to hit a limit when handling a modest Maildir case; does this 
reflect a bug?

rsync'ing to a new mail server, after fewer than 100,000 files there are 
intermittent failures:

  rsync: [receiver] open "/home/mark/Maildir/.robot/cur/1633731549.M990187P7732.yello.[redacted],S=69473,W=70413:2," failed: No space left on device (28)
  rsync: [receiver] rename "/home/mark/Maildir/.robot/cur/.1624626598.M748388P84607.yello.[redacted],S=17049,W=17352:2,.oBphKA" -> ".robot/cur/1624626598.M748388P84607.yello.[redacted],S=17049,W=17352:2,": No space left on device (28)

The kernel:

  EXT4-fs warning (device dm-4): ext4_dx_add_entry:2351: Directory (ino: 225811) index full, reach max htree level :2
  EXT4-fs warning (device dm-4): ext4_dx_add_entry:2355: Large directory feature is not enabled on this filesystem

Reaching for 'large_dir' seems premature as this feature is reported as 
useful for 10M+ files, but this is much lower.

A 'bad' filename will fail consistently. Assuming the 10M+ absolute limit, 
is the tree grossly imbalanced?

Intuitively, 'htree level :2' does not sound particular deep.

The source folder is 195,000 files -- large, but not crazy. rsync 
eventually hit a ceiling having written 177,482 of the files. I can still 
create new ones on the command line with non-Maildir names.

Ruled out quotas, by disabling them with "tune2fs -O ^quota" and 

See below for additional info.


$ uname -a
Linux floyd 5.10.78-0-virt #1-Alpine SMP Thu, 11 Nov 2021 14:31:09 +0000 x86_64 GNU/Linux

$ mke2fs -q -t ext4 /dev/vg0/home

$ rsync -va --exclude 'dovecot*' yello:Maildir/. $HOME/Maildir

$ ls | head -15

$ touch abc

$ touch 1624626598.M748388P84607.yello.[redacted],S=17049,W=17352:2,
touch: cannot touch '1624626598.M748388P84607.yello.[redacted],S=17049,W=17352:2,': No space left on device

$ dumpe2fs /dev/vg0/home
Filesystem volume name:   <none>
Last mounted on:          /home
Filesystem UUID:          ad26c968-d057-4d44-bef9-1e2df347580e
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              5225472
Block count:              21229568
Reserved block count:     851459
Overhead clusters:        22361
Free blocks:              8058180
Free inodes:              4799979
First block:              1
Block size:               1024
Fragment size:            1024
Group descriptor size:    64
Reserved GDT blocks:      96
Blocks per group:         8192
Fragments per group:      8192
Inodes per group:         2016
Inode blocks per group:   504
Flex block group size:    16
Filesystem created:       Mon Nov  8 13:14:56 2021
Last mount time:          Fri Nov 12 18:43:14 2021
Last write time:          Fri Nov 12 18:43:14 2021
Mount count:              27
Maximum mount count:      -1
Last checked:             Mon Nov  8 13:14:56 2021
Check interval:           0 (<none>)
Lifetime writes:          14 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     32
Desired extra isize:      32
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      839d2871-b97e-456d-9724-096db15931b8
Journal backup:           inode blocks
Checksum type:            crc32c
Checksum:                 0x5974a8b1
Journal features:         journal_incompat_revoke journal_64bit 
Total journal size:       4096k
Total journal blocks:     4096
Max transaction length:   4096
Fast commit length:       0
Journal sequence:         0x00000a2a
Journal start:            702
Journal checksum type:    crc32c
Journal checksum:         0x4d693e79

Powered by blists - more mailing lists