linux-ext4 - [RFC] improve malloc for large filesystems

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:   Tue, 21 Apr 2020 08:22:29 +0000
From:   Alex Zhuravlev <azhuravlev@...mcloud.com>
To:     "linux-ext4@...r.kernel.org" <linux-ext4@...r.kernel.org>
Subject: [RFC] improve malloc for large filesystems - limit scanning

Hi, yet another patch.

There is an issue with current mballoc on huge filesystems - when a small allocation is requested, mballoc turns it into a group preallocation (512 blocks by default),
then scans all bitmaps to find this good chunk. The problem is that stages 0 and 1 are optimised for CPU cycles and skip not-interesting groups, but to learn
the group is not interesting it should be initialised. If it’s not then IO is required. If the filesystem is highly fragmented then it may take hours (literally) to find this good
chunk. The previous prefetching patch improves this, but still leads to lots of IO and very expensive allocations in some cases.
This patch changes mballoc in a different way - as stages 0/1 are optimisations, we can skip them for non-initialised groups (e.g. right after mount) and just scan
bitmaps for all available blocks (stage 2).

Please review.

Thanks, Alex

Download attachment "0001-mballoc-limit-scanning-of-uninitialized-groups.patch" of type "application/octet-stream" (1695 bytes)