lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <aLsQQQlq_j3ziajp@li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com>
Date: Fri, 5 Sep 2025 22:00:57 +0530
From: Ojaswin Mujoo <ojaswin@...ux.ibm.com>
To: John Garry <john.g.garry@...cle.com>
Cc: Zorro Lang <zlang@...hat.com>, fstests@...r.kernel.org,
        Ritesh Harjani <ritesh.list@...il.com>, djwong@...nel.org,
        tytso@....edu, linux-xfs@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-ext4@...r.kernel.org
Subject: Re: [PATCH v5 07/12] generic: Add atomic write multi-fsblock
 O_[D]SYNC tests

On Tue, Sep 02, 2025 at 04:14:45PM +0100, John Garry wrote:
> On 22/08/2025 09:02, Ojaswin Mujoo wrote:
> > This adds various atomic write multi-fsblock stresst tests
> 
> stress
> 
> > with mixed mappings and O_SYNC, to ensure the data and metadata
> > is atomically persisted even if there is a shutdown.
> > 
> > Suggested-by: Ritesh Harjani (IBM) <ritesh.list@...il.com>
> > Reviewed-by: Darrick J. Wong <djwong@...nel.org>
> > Signed-off-by: Ojaswin Mujoo <ojaswin@...ux.ibm.com>
> 
> Regardless of any comments made (which are minor):
> 
> Reviewed-by: John Garry <john.g.garry@...cle.com>
> 
> > ---
> >   tests/generic/1228     | 137 +++++++++++++++++++++++++++++++++++++++++
> >   tests/generic/1228.out |   2 +
> >   2 files changed, 139 insertions(+)
> >   create mode 100755 tests/generic/1228
> >   create mode 100644 tests/generic/1228.out
> > 
> > diff --git a/tests/generic/1228 b/tests/generic/1228
> > new file mode 100755
> > index 00000000..888599ce
> > --- /dev/null
> > +++ b/tests/generic/1228
> > @@ -0,0 +1,137 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2025 IBM Corporation. All Rights Reserved.
> > +#
> > +# FS QA Test 1228
> > +#
> > +# Atomic write multi-fsblock data integrity tests with mixed mappings
> > +# and O_SYNC
> > +#
> > +. ./common/preamble
> > +. ./common/atomicwrites
> > +_begin_fstest auto quick rw atomicwrites
> > +
> > +_require_scratch_write_atomic_multi_fsblock
> > +_require_atomic_write_test_commands
> > +_require_scratch_shutdown
> > +_require_xfs_io_command "truncate"
> > +
> > +_scratch_mkfs >> $seqres.full
> > +_scratch_mount >> $seqres.full
> > +
> > +check_data_integrity() {
> > +	actual=$(_hexdump $testfile)
> > +	if [[ "$expected" != "$actual" ]]
> > +	then
> > +		echo "Integrity check failed"
> > +		echo "Integrity check failed" >> $seqres.full
> > +		echo "# Expected file contents:" >> $seqres.full
> > +		echo "$expected" >> $seqres.full
> > +		echo "# Actual file contents:" >> $seqres.full
> > +		echo "$actual" >> $seqres.full
> > +
> > +		_fail "Data integrity check failed. The atomic write was torn."
> > +	fi
> > +}
> > +
> > +prep_mixed_mapping() {
> > +	$XFS_IO_PROG -c "truncate 0" $testfile >> $seqres.full
> > +	local off=0
> > +	local mapping=""
> > +
> > +	local operations=("W" "H" "U")
> > +	local num_blocks=$((awu_max / blksz))
> > +	for ((i=0; i<num_blocks; i++)); do
> > +		local index=$((RANDOM % ${#operations[@]}))
> > +		local map="${operations[$index]}"
> > +		local mapping="${mapping}${map}"
> > +
> > +		case "$map" in
> > +			"W")
> > +				$XFS_IO_PROG -dc "pwrite -S 0x61 -b $blksz $off $blksz" $testfile > /dev/null
> > +				;;
> > +			"H")
> > +				# No operation needed for hole
> > +				;;
> > +			"U")
> > +				$XFS_IO_PROG -c "falloc $off $blksz" $testfile >> /dev/null
> > +				;;
> > +		esac
> > +		off=$((off + blksz))
> > +	done
> > +
> > +	echo "+ + Mixed mapping prep done. Full mapping pattern: $mapping" >> $seqres.full
> > +
> > +	sync $testfile
> > +}
> > +
> > +verify_atomic_write() {
> > +	test $bytes_written -eq $awu_max || _fail "atomic write len=$awu_max assertion failed"
> > +	check_data_integrity
> > +}
> > +
> > +mixed_mapping_test() {
> > +	prep_mixed_mapping
> > +
> > +	echo -"+ + Performing O_DSYNC atomic write from 0 to $awu_max" >> $seqres.full
> > +	if [[ "$1" == "shutdown" ]]
> > +	then
> > +		bytes_written=$($XFS_IO_PROG -x -dc \
> > +				"pwrite -DA -V1 -b $awu_max 0 $awu_max" \
> > +				-c "shutdown" $testfile | grep wrote | \
> > +				awk -F'[/ ]' '{print $2}')
> > +		_scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed"
> > +	else
> > +		bytes_written=$($XFS_IO_PROG -dc \
> > +				"pwrite -DA -V1 -b $awu_max 0 $awu_max" $testfile | \
> > +				grep wrote | awk -F'[/ ]' '{print $2}')
> > +	fi
> > +
> > +	verify_atomic_write
> > +}
> > +
> > +testfile=$SCRATCH_MNT/testfile
> > +touch $testfile
> > +
> > +awu_max=$(_get_atomic_write_unit_max $testfile)
> > +blksz=$(_get_block_size $SCRATCH_MNT)
> > +
> > +# Create an expected pattern to compare with
> > +$XFS_IO_PROG -tc "pwrite -b $awu_max 0 $awu_max" $testfile >> $seqres.full
> > +expected=$(_hexdump $testfile)
> > +echo "# Expected file contents:" >> $seqres.full
> > +echo "$expected" >> $seqres.full
> > +echo >> $seqres.full
> > +
> > +echo "# Test 1: Do O_DSYNC atomic write on random mixed mapping:" >> $seqres.full
> > +echo >> $seqres.full
> > +for ((iteration=1; iteration<=10; iteration++)); do
> > +	echo "=== Mixed Mapping Test Iteration $iteration ===" >> $seqres.full
> > +
> > +	echo "+ Testing without shutdown..." >> $seqres.full
> > +	mixed_mapping_test
> > +	echo "Passed!" >> $seqres.full
> > +
> > +	echo "+ Testing with sudden shutdown..." >> $seqres.full
> > +	mixed_mapping_test "shutdown"
> > +	echo "Passed!" >> $seqres.full
> > +
> > +	echo "Iteration $iteration completed: OK" >> $seqres.full
> > +	echo >> $seqres.full
> > +done
> > +echo "# Test 1: Do O_SYNC atomic write on random mixed mapping (10 iterations): OK" >> $seqres.full
> 
> big nit: you should keep the loop count in a global variable

I believe you mean something like:

  echo " ... ($iters iteration): OK" 

If so, sure I agree. I'll make the changes.

Thanks for the review!

Regards,
ojaswin
> 
> 
> > +
> > +
> > +echo >> $seqres.full
> > +echo "# Test 2: Do extending O_SYNC atomic writes: " >> $seqres.full
> > +bytes_written=$($XFS_IO_PROG -x -dstc "pwrite -A -V1 -b $awu_max 0 $awu_max" \
> > +		-c "shutdown" $testfile | grep wrote | awk -F'[/ ]' '{print $2}')
> > +_scratch_cycle_mount >>$seqres.full 2>&1 || _fail "remount failed"
> > +verify_atomic_write
> > +echo "# Test 2: Do extending O_SYNC atomic writes: OK" >> $seqres.full
> > +
> > +# success, all done
> > +echo "Silence is golden"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/generic/1228.out b/tests/generic/1228.out
> > new file mode 100644
> > index 00000000..1baffa91
> > --- /dev/null
> > +++ b/tests/generic/1228.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 1228
> > +Silence is golden
> 

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ