linux-kernel - Re: fiemap is slow on btrfs on files with multiple extents

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20220805125231.9327.409509F4@e16-tech.com>
Date:   Fri, 05 Aug 2022 12:52:32 +0800
From:   Wang Yugui <wangyugui@...-tech.com>
To:     Josef Bacik <josef@...icpanda.com>
Cc:     Pavel Tikhomirov <ptikhomirov@...tuozzo.com>,
        Chris Mason <clm@...com>, David Sterba <dsterba@...e.com>,
        linux-btrfs@...r.kernel.org, lkml <linux-kernel@...r.kernel.org>,
        Chen Liang-Chun <featherclc@...il.com>,
        Alexander Mikhalitsyn <alexander.mikhalitsyn@...tuozzo.com>,
        kernel@...nvz.org,
        Dominique MARTINET <dominique.martinet@...ark-techno.com>,
        Yu Kuai <yukuai3@...wei.com>, Theodore Ts'o <tytso@....edu>
Subject: Re: fiemap is slow on btrfs on files with multiple extents

Hi,

> On Thu, Aug 04, 2022 at 07:30:52PM +0300, Pavel Tikhomirov wrote:
> > I ran the below test on Fedora 36 (the test basically creates "very" sparse
> > file, with 4k data followed by 4k hole again and again for the specified
> > length and uses fiemap to count extents in this file) and face the problem
> > that fiemap hangs for too long (for instance comparing to ext4 version).
> > Fiemap with 32768 extents takes ~37264 us and with 65536 extents it takes
> > ~34123954 us, which is x1000 times more when file only increased twice the
> > size:
> >
> 
> Ah that was helpful, thank you.  I think I've spotted the problem, please give
> this a whirl to make sure we're seeing the same thing.  Thanks,
> 
> Josef

This patch improve the performance very well, but  it seems to break
xfstest generic/285.

xfstest generic/285:
06.11 SEEK_HOLE expected 8192 or 16384, got 8191.                 FAIL
06.12 SEEK_DATA expected 8191 or 8191, got 12288.                 FAIL
06.23 SEEK_HOLE expected 16384 or 16384, got 16383.               FAIL
06.24 SEEK_DATA expected 16383 or 16383, got -1.                  FAIL

Best Regards
Wang Yugui (wangyugui@...-tech.com)
2022/08/05


>  
> From 1133d5ebf952ebf334bc7be21a575b1f52eb71d4 Mon Sep 17 00:00:00 2001
> Message-Id: <1133d5ebf952ebf334bc7be21a575b1f52eb71d4.1659638886.git.josef@...icpanda.com>
> From: Josef Bacik <josef@...icpanda.com>
> Date: Thu, 4 Aug 2022 14:45:53 -0400
> Subject: [PATCH] btrfs: don't search entire range for delalloc with fiemap
> 
> For the case where we have
> 
> [EXTENT1][HOLE][EXTENT2]
> 
> If we fiemap from [HOLE] we will search to len (which could be -1) to
> see if there's any delalloc extents in the range, however in the above
> case btrfs_get_extent() returns a hole em for just the range of the
> hole, as it will find EXTENT2, so all we need to do is search for
> delalloc in the hole range, not the entire rest of the requested fiemap
> range.
> 
> This fixes the extremely bad fiemap performance with very large sparse
> files.
> 
> Signed-off-by: Josef Bacik <josef@...icpanda.com>
> ---
>  fs/btrfs/inode.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
> index 8fc1e3b6e00c..b7ad8f7a7b53 100644
> --- a/fs/btrfs/inode.c
> +++ b/fs/btrfs/inode.c
> @@ -7095,7 +7095,7 @@ struct extent_map *btrfs_get_extent_fiemap(struct btrfs_inode *inode,
>  		hole_em = em;
>  
>  	/* check to see if we've wrapped (len == -1 or similar) */
> -	end = start + len;
> +	end = em->start + em->len;
>  	if (end < start)
>  		end = (u64)-1;
>  	else
> -- 
> 2.36.1