linux-ext4 - Re: bug with large

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 28 Jul 2021 16:56:05 +0300
From:   Благодаренко Артём 
        <artem.blagodarenko@...il.com>
To:     Carlos Carvalho <carlos@...ica.ufpr.br>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: bug with large_dir in 5.12.17

Hello,

I have reproduced the issue on the master head

# git rev-parse HEAD
f158f8962ed7e884fa168f354c488f3afa3eb6db

Here is a reproducer. Not perfect bash script, but reproduce the problem well

#!/bin/bash  
dir_path=/mnt/ext4/
cd $dir_path
i=0
target_file=$dir_path/foofile$i
touch $target_file
while test $i -lt 100000000 ; do
        if test $((i % 40000)) -eq 0; then
                echo "$i names created"
                target_file=$dir_path/foofile$i
                touch $target_file
        fi
    lncom=`printf "ln $target_file n%0253u\n" $i`
    `$lncom`
    i=$((i + 1))
done

Without large_dir option 1504565 files created and then I got no space

1480000 names created
ln: failed to create hard link 'n0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001504566': No space left on device
ln: failed to create hard link

[root@...2 ~]# ls /mnt/ext4/ | wc -l
1510070
[root@...2 ~]#

 
When I enabled large_dir and used the same script, but started from I=1600000


Some amount (36526) of names were created and finally  this error happened.

sh ~/filldir.sh
1600000 names created
ln: failed to create hard link 'n0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001636527' => '/mnt/ext4//foofile1600000': Structure needs cleaning
ln: failed to access 'n0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001636797': Structure needs cleaning

From dimes

[27927.219844] EXT4-fs (loop1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[28058.315171] EXT4-fs error (device loop1): dx_probe:887: inode #2: block 147359: comm ln: directory leaf block found instead of index block
[28059.414053] EXT4-fs error (device loop1): dx_probe:887: inode #2: block 147359: comm ln: directory leaf block found instead of index block
[28059.627589] EXT4-fs error (device loop1): dx_probe:887: inode #2: block 147359: comm ln: directory leaf block found instead of index block

>are you able to reproduce this error on a new directory that did not hit
the 2-level htree limit before enabling large_dir, or did you only see this
with directories that hit the 2-level htree limit before the update?

I am executing this test now. Will report shortly.

Best regards,
Artem Blagodarenko
> On 22 Jul 2021, at 17:23, Carlos Carvalho <carlos@...ica.ufpr.br> wrote:
> 
> There is a bug when enabling large_dir in 5.12.17. I got this during a backup:
> 
> index full, reach max htree level :2
> Large directory feature is not enabled on this filesystem
> 
> So I unmounted, ran tune2fs -O large_dir /dev/device and mounted again. However
> this error appeared:
> 
> dx_probe:864: inode #576594294: block 144245: comm rsync: directory leaf block found instead of index block
> 
> I unmounted, ran fsck and it "salvaged" a bunch of directories. However at the
> next backup run the same errors appeared again.
> 
> This is with vanilla 5.2.17.