linux-ext4 - Re: some large dir testing results

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 21 Apr 2017 17:11:43 +0300
From:   Alexey Lyashkov <alexey.lyashkov@...il.com>
To:     Bernd Schubert <bschubert@....com>
Cc:     linux-ext4 <linux-ext4@...r.kernel.org>,
        Andreas Dilger <adilger@...ger.ca>,
        Artem Blagodarenko <artem.blagodarenko@...il.com>
Subject: Re: some large dir testing results


> 21 апр. 2017 г., в 17:08, Bernd Schubert <bschubert@....com> написал(а):
>> 
>> Initial analyze say about several problems
>> 0) CPU load isn’t high, and perf top say ldiskfs functions isn’t hot (2%-3%
>> cpu), most spent for dir entry checking function.
>> 
>> 1) lookup have a large time to read a directory block to verify file not
>> exist. I think it because a block fragmentation. [root@...k03 ~]# cat
>> /proc/100993/stack
>> [<ffffffff81211b1e>] sleep_on_buffer+0xe/0x20
>> [<ffffffff812130da>] __wait_on_buffer+0x2a/0x30
>> [<ffffffffa0899e6c>] ldiskfs_bread+0x7c/0xc0 [ldiskfs]
>> [<ffffffffa088ee4a>] __ldiskfs_read_dirblock+0x4a/0x400 [ldiskfs]
>> [<ffffffffa08915af>] ldiskfs_dx_find_entry+0xef/0x200 [ldiskfs]
>> [<ffffffffa0891b8b>] ldiskfs_find_entry+0x4cb/0x570 [ldiskfs]
>> [<ffffffffa08921d5>] ldiskfs_lookup+0x75/0x230 [ldiskfs]
>> [<ffffffff811e8e7d>] lookup_real+0x1d/0x50
>> [<ffffffff811e97f2>] __lookup_hash+0x42/0x60
>> [<ffffffff811ee848>] filename_create+0x98/0x180
>> [<ffffffff811ef6e1>] user_path_create+0x41/0x60
>> [<ffffffff811f084a>] SyS_mknodat+0xda/0x220
>> [<ffffffff811f09ad>] SyS_mknod+0x1d/0x20
>> [<ffffffff81645549>] system_call_fastpath+0x16/0x1b
>> [<ffffffffffffffff>] 0xffffffffffffffff
> 
> I wrote patches for ext4 a long time ago to get a better caching for that
> 
> https://patchwork.ozlabs.org/patch/101200/
> 
> 
> For FhGFS/BeeGFS we then decided to use a totally different directory layout, 
> which totally eliminated the underlying issue for the main requirement or 
> large dirs at all. (Personally I would recommend to do the something similar 
> for Lustre - using hash dirs to store objects has a much too random access 
> pattern once the file system gets used with many files...).
> 
> Also, a caching issue has been fixed by Mel Gorman in 3.11 (I didn't check if 
> these patches are backported to any vendor kernel).
> 
> 
Bernd,

Thanks to point we to patches, I will test with it on my next test loop.
As about a different layout - it’s exist as separate option.


Alex