linux-kernel - Re: fiemap is slow on btrfs on files with multiple extents

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YuzI7Tqi3n+d+V+P@atmark-techno.com>
Date:   Fri, 5 Aug 2022 16:38:21 +0900
From:   Dominique MARTINET <dominique.martinet@...ark-techno.com>
To:     Pavel Tikhomirov <ptikhomirov@...tuozzo.com>,
        Josef Bacik <josef@...icpanda.com>
Cc:     Chris Mason <clm@...com>, David Sterba <dsterba@...e.com>,
        linux-btrfs@...r.kernel.org, lkml <linux-kernel@...r.kernel.org>,
        Chen Liang-Chun <featherclc@...il.com>,
        Alexander Mikhalitsyn <alexander.mikhalitsyn@...tuozzo.com>,
        kernel@...nvz.org, Yu Kuai <yukuai3@...wei.com>,
        Theodore Ts'o <tytso@....edu>
Subject: Re: fiemap is slow on btrfs on files with multiple extents

Pavel Tikhomirov wrote on Thu, Aug 04, 2022 at 07:30:52PM +0300:
> I see a similar problem here
> https://lore.kernel.org/linux-btrfs/Yr4nEoNLkXPKcOBi@atmark-techno.com/#r ,
> but in my case I have "5.18.6-200.fc36.x86_64" fedora kernel which does not
> have 5ccc944dce3d ("filemap: Correct the conditions for marking a folio as
> accessed") commit, so it should be something else.

The root cause might be different but I guess they're related enough: if
fiemap gets faster enough even when the whole file is in cache I guess
that works for me :)

Josef Bacik wrote on Thu, Aug 04, 2022 at 02:49:39PM -0400:
> On Thu, Aug 04, 2022 at 07:30:52PM +0300, Pavel Tikhomirov wrote:
> > I ran the below test on Fedora 36 (the test basically creates "very" sparse
> > file, with 4k data followed by 4k hole again and again for the specified
> > length and uses fiemap to count extents in this file) and face the problem
> > that fiemap hangs for too long (for instance comparing to ext4 version).
> > Fiemap with 32768 extents takes ~37264 us and with 65536 extents it takes
> > ~34123954 us, which is x1000 times more when file only increased twice the
> > size:
> >
> 
> Ah that was helpful, thank you.  I think I've spotted the problem, please give
> this a whirl to make sure we're seeing the same thing.  Thanks,

FWIW this patch does help a tiny bit, but I'm still seeing a huge
slowdown: with patch cp goes from ~600MB/s (55s) to 136MB/s (3m55s) on
the second run; and without the patch I'm getting 47s and 5m35
respectively so this has gotten a bit better but these must still be
cases running through the whole list (e.g. when not hitting a hole?)


My reproducer is just running 'cp file /dev/null' twice on a file with
194955 extents (same file with mixed compressed & non-compressed extents
as last time), so should be close enough to what Pavel was describing in
just much worse.

-- 
Dominique