linux-ext4 - Re: [PATCH] ext4: fix partial cluster handling for bigalloc file systems

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 12 Mar 2014 17:41:22 -0400
From:	Eric Whitney <enwlinux@...il.com>
To:	Eric Whitney <enwlinux@...il.com>
Cc:	linux-ext4@...r.kernel.org, tytso@....edu
Subject: Re: [PATCH] ext4: fix partial cluster handling for bigalloc file
 systems

* Eric Whitney <enwlinux@...il.com>:
> Commit 9cb00419fa, which enables hole punching for bigalloc file
> systems, exposed a bug introduced by commit 6ae06ff51e in an earlier
> release.  When run on a bigalloc file system, xfstests generic/013, 068,
> 075, 083, 091, 100, 112, 127, 263, 269, and 270 fail with e2fsck errors
> or cause kernel error messages indicating that previously freed blocks
> are being freed again.
> 
> The latter commit optimizes the selection of the starting extent in
> ext4_ext_rm_leaf() when hole punching by beginning with the extent
> supplied in the path argument rather than with the last extent in the
> leaf node (as is still done when truncating).  However, the code in
> rm_leaf that initially sets partial_cluster to track cluster sharing on
> extent boundaries is only guaranteed to run if rm_leaf starts with the
> last node in the leaf.  Consequently, partial_cluster is not correctly
> initialized when hole punching, and a cluster on the boundary of a
> punched region that should be retained may instead be deallocated.
> 
> Signed-off-by: Eric Whitney <enwlinux@...il.com>
> ---
>  fs/ext4/extents.c | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
> 
> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
> index 74bc2d5..f9459cb 100644
> --- a/fs/ext4/extents.c
> +++ b/fs/ext4/extents.c
> @@ -2585,7 +2585,27 @@ ext4_ext_rm_leaf(handle_t *handle, struct inode *inode,
>  	ex_ee_block = le32_to_cpu(ex->ee_block);
>  	ex_ee_len = ext4_ext_get_actual_len(ex);
>  
> +	/*
> +	 * If we're starting with an extent other than the last one in the
> +	 * node, we need to see if it shares a cluster with the extent to
> +	 * the right (towards the end of the file). If its leftmost cluster
> +	 * is this extent's rightmost cluster and it is not cluster aligned,
> +	 * we'll mark it as a partial that is not to be deallocated.
> +	 */
> +
> +	if (ex != EXT_LAST_EXTENT(eh)) {
> +		ext4_fsblk_t current_pblk, right_pblk;
> +		long long current_cluster, right_cluster;
> +
> +		current_pblk = ext4_ext_pblock(ex) + ex_ee_len - 1;
> +		current_cluster = (long long)EXT4_B2C(sbi, current_pblk);
> +		right_pblk = ext4_ext_pblock(ex + 1);
> +		right_cluster = (long long)EXT4_B2C(sbi, right_pblk);
> +		if (current_cluster == right_cluster &&
> +			EXT4_PBLK_COFF(sbi, right_pblk))
> +			*partial_cluster = -right_cluster;
> +	}
> +
>  	trace_ext4_ext_rm_leaf(inode, start, ex, *partial_cluster);
>  
>  	while (ex >= EXT_FIRST_EXTENT(eh) &&
> -- 
> 1.8.3.2
> 

This patch fixes all 11 of the new test failures I've seen when regressing
various 3.14-rc's on bigalloc in comparison with 3.13 final.  Elsewhere,
I'd mentioned that I had a patch that fixed 7 out of the 11, but addressing a
small bug of mine got the rest of them.  So, this fix should be complete.

Please note that it's still possible to see e2fsck failures and kernel error
messages similar to those associated with these 11 tests when running on
bigalloc.  In particular, running generic/300 and 311 and shared/298 can
result in kernel error messages like:

ext4_mb_free_metadata:4554: group 0, block 19072:Block already on to-be-freed list

However, I've reverted the root cause (commit 6ae06ff51e), and still get those
messages, so I don't think they represent cases not covered by my patch -
just more problems to pursue.

Thanks,
Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html