linux-ext4 - Re: [RFC,2/2] xfstests: common/rc: add cluster size support for ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20210706181452.GB11571@locust>
Date:   Tue, 6 Jul 2021 11:14:52 -0700
From:   "Darrick J. Wong" <djwong@...nel.org>
To:     jefflexu@...ux.alibaba.com
Cc:     "Darrick J. Wong" <darrick.wong@...cle.com>,
        fstests@...r.kernel.org, linux-ext4@...r.kernel.org,
        linux-xfs@...r.kernel.org
Subject: Re: [RFC,2/2] xfstests: common/rc: add cluster size support for ext4

On Mon, Jul 05, 2021 at 02:58:51PM +0800, JeffleXu wrote:
> 
> Sorry for digging this really old post [1]. The overall background is
> that, @offset and @len need to be aligned with cluster size when doing
> fallocate(), or several xfstests cases calling fsx will fail if the
> tested filesystem enabling 'bigalloc' feature.
> 
> On April 27, 2020, 5:33 p.m. UTC Darrick J. Wong wrote:
> 
> > On Fri, Apr 24, 2020 at 05:33:50PM +0800, Jeffle Xu wrote:
> >> Inserting and collapsing range on ext4 with 'bigalloc' feature will
> >> fail due to the offset and size should be alligned with the cluster
> >> size.
> >> 
> >> The previous patch has add support for cluster size in fsx. Detect and
> >> pass the cluster size parameter to fsx if the underlying filesystem
> >> is ext4 with bigalloc.
> >> 
> >> Signed-off-by: Jeffle Xu <jefflexu@...ux.alibaba.com>
> >> ---
> >>  common/rc | 9 +++++++++
> >>  1 file changed, 9 insertions(+)
> >> 
> >> diff --git a/common/rc b/common/rc
> >> index 2000bd9..71dde5f 100644
> >> --- a/common/rc
> >> +++ b/common/rc
> >> @@ -3908,6 +3908,15 @@ run_fsx()
> >>  {
> >>  	echo fsx $@
> >>  	local args=`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psize /g"`
> >> +
> >> +	if [ "$FSTYP" == "ext4" ]; then
> >> +		local cluster_size=$(tune2fs -l $TEST_DEV | grep 'Cluster size' | awk '{print $3}')
> >> +		if [ -n $cluster_size ]; then
> >> +			echo "cluster size: $cluster_size"
> >> +			args="$args -u $cluster_size"
> >> +		fi
> >> +	fi
> > 
> > Computing the file allocation block size ought to be a separate helper.
> > 
> > I wonder if there's a standard way to report cluster sizes, seeing as
> > fat, ext4, ocfs2, and xfs can all have minimum space allocation units
> > that are larger than the base fs block size.
> 
> In fact only for insert_range and collapse range of ext4 and xfs (in
> realtime mode), @offset and @len need to be aligned with cluster size.
> 
> Though fat and ocfs2 also support cluster size, ocfs2 only supports
> preallocate and punch_hole, and fat only supports preallocate, in which
> case @offset and @len needn't be aligned with cluster size.
> 
> 
> So we need to align @offset and @len with cluster size only for ext4 and
> xfs (in realtime mode) at a minimum cost, to fix this issue. But the
> question is, there's no standard programming interface exporting cluster
> size. For both ext4 and xfs, it's stored as a binary data in disk
> version superblock, e.g., tune2fs could detect the cluster size of ext4.
> 
> 
> Any idea on how to query the cluster size?

xfs and ocfs2 return the rt extent size in stat.st_blksize.

In fstestsland you could use _get_file_block_size to figure out the
allocation unit.

Alternately, I've been testing a more permanent fix for the blocksize
issues[1]; perhaps that will fix the problem?

(Note that the series is at the end of my dev tree, so it's likely to
have apply errors for the tests that exist in djwong-dev but aren't
upstream.)

--D

[1] https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=check-blocksize-congruency

> 
> 
> [1]
> https://patchwork.kernel.org/project/fstests/cover/1587720830-11955-1-git-send-email-jefflexu@linux.alibaba.com/
> 
> -- 
> Thanks,
> Jeffle