| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <aKbb2XhcsMMhBlgb@li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com>
Date: Thu, 21 Aug 2025 14:12:01 +0530
From: Ojaswin Mujoo <ojaswin@...ux.ibm.com>
To: John Garry <john.g.garry@...cle.com>
Cc: Zorro Lang <zlang@...hat.com>, fstests@...r.kernel.org,
Ritesh Harjani <ritesh.list@...il.com>, djwong@...nel.org,
tytso@....edu, linux-xfs@...r.kernel.org, linux-kernel@...r.kernel.org,
linux-ext4@...r.kernel.org
Subject: Re: [PATCH v4 04/11] generic: Add atomic write test using fio crc
check verifier
On Wed, Aug 13, 2025 at 02:39:40PM +0100, John Garry wrote:
> On 10/08/2025 14:41, Ojaswin Mujoo wrote:
> > This adds atomic write test using fio based on it's crc check verifier.
> > fio adds a crc for each data block. If the underlying device supports
> > atomic write then it is guaranteed that we will never have a mix data from
> > two threads writing on the same physical block.
> >
> > Avoid doing overlapping parallel atomic writes because it might give
> > unexpected results. Use offset_increment=, size= fio options to achieve
> > this behavior.
> >
>
> You are not really describing what the test does.
>
> In the first paragraph, you state what fio verify function does and then
> describe what RWF_ATOMIC means when we only use HW support, i.e. serialises.
> In the second you mention that we guarantee no inter-thread overlapping
> writes.
Got it John, I will add better commit messages for the fio tests.
>
> From a glance at the code below, in this test each thread writes to a
> separate part of the file and then verifies no crc corruption. But even with
> atomic=0, I would expect no corruption here.
Right, this is mostly a stress test that is ensuring that all the new
atomic write code paths are not causing anything to break or
introducing any regressions. This should pass with both atomic or non
atomic writes but by using RWF_ATOMIC we excercise the atomic specific
code paths, improving the code coverage.
Regards,
ojaswin
>
> Thanks,
> John
>
> > Co-developed-by: Ritesh Harjani (IBM) <ritesh.list@...il.com>
> > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@...il.com>
> > Signed-off-by: Ojaswin Mujoo <ojaswin@...ux.ibm.com>
> > ---
> > tests/generic/1226 | 107 +++++++++++++++++++++++++++++++++++++++++
> > tests/generic/1226.out | 2 +
> > 2 files changed, 109 insertions(+)
> > create mode 100755 tests/generic/1226
> > create mode 100644 tests/generic/1226.out
> >
> > diff --git a/tests/generic/1226 b/tests/generic/1226
> > new file mode 100755
> > index 00000000..efc360e1
> > --- /dev/null
> > +++ b/tests/generic/1226
> > @@ -0,0 +1,107 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2025 IBM Corporation. All Rights Reserved.
> > +#
> > +# FS QA Test 1226
> > +#
> > +# Validate FS atomic write using fio crc check verifier.
> > +#
> > +. ./common/preamble
> > +. ./common/atomicwrites
> > +
> > +_begin_fstest auto aio rw atomicwrites
> > +
> > +_require_scratch_write_atomic
> > +_require_odirect
> > +_require_aio
> > +
> > +_scratch_mkfs >> $seqres.full 2>&1
> > +_scratch_mount
> > +_require_xfs_io_command "falloc"
> > +
> > +touch "$SCRATCH_MNT/f1"
> > +awu_min_write=$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1")
> > +awu_max_write=$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1")
> > +
> > +blocksize=$(_max "$awu_min_write" "$((awu_max_write/2))")
> > +threads=$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100")
> > +filesize=$((blocksize * threads * 100))
> > +depth=$threads
> > +io_size=$((filesize / threads))
> > +io_inc=$io_size
> > +testfile=$SCRATCH_MNT/test-file
> > +
> > +fio_config=$tmp.fio
> > +fio_out=$tmp.fio.out
> > +
> > +fio_aw_config=$tmp.aw.fio
> > +fio_verify_config=$tmp.verify.fio
> > +
> > +function create_fio_configs()
> > +{
> > + create_fio_aw_config
> > + create_fio_verify_config
> > +}
> > +
> > +function create_fio_verify_config()
> > +{
> > +cat >$fio_verify_config <<EOF
> > + [verify-job]
> > + direct=1
> > + ioengine=libaio
> > + rw=read
> > + bs=$blocksize
> > + filename=$testfile
> > + size=$filesize
> > + iodepth=$depth
> > + group_reporting=1
> > +
> > + verify_only=1
> > + verify=crc32c
> > + verify_fatal=1
> > + verify_state_save=0
> > + verify_write_sequence=0
> > +EOF
> > +}
> > +
> > +function create_fio_aw_config()
> > +{
> > +cat >$fio_aw_config <<EOF
> > + [atomicwrite-job]
> > + direct=1
> > + ioengine=libaio
> > + rw=randwrite
> > + bs=$blocksize
> > + filename=$testfile
> > + size=$io_inc
> > + offset_increment=$io_inc
> > + iodepth=$depth
> > + numjobs=$threads
> > + group_reporting=1
> > + atomic=1
> > +
> > + verify_state_save=0
> > + verify=crc32c
> > + do_verify=0
> > +EOF
> > +}
> > +
> > +create_fio_configs
> > +_require_fio $fio_aw_config
> > +
> > +cat $fio_aw_config >> $seqres.full
> > +cat $fio_verify_config >> $seqres.full
> > +
> > +$XFS_IO_PROG -fc "falloc 0 $filesize" $testfile >> $seqres.full
> > +
> > +$FIO_PROG $fio_aw_config >> $seqres.full
> > +ret1=$?
> > +$FIO_PROG $fio_verify_config >> $seqres.full
> > +ret2=$?
> > +
> > +[[ $ret1 -eq 0 && $ret2 -eq 0 ]] || _fail "fio with atomic write failed"
> > +
> > +# success, all done
> > +echo Silence is golden
> > +status=0
> > +exit
> > diff --git a/tests/generic/1226.out b/tests/generic/1226.out
> > new file mode 100644
> > index 00000000..6dce0ea5
> > --- /dev/null
> > +++ b/tests/generic/1226.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 1226
> > +Silence is golden
>
Powered by blists - more mailing lists