linux-ext4 - Re: [PATCH v4 0/5] check: add option to rerun failed tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20220708163654.7luwvjfdn4ep5jgw@zlang-mailbox>
Date:   Sat, 9 Jul 2022 00:36:54 +0800
From:   Zorro Lang <zlang@...hat.com>
To:     David Disseldorp <ddiss@...e.de>
Cc:     fstests@...r.kernel.org, tytso@....edu, djwong@...nel.org,
        linux-ext4@...r.kernel.org
Subject: Re: [PATCH v4 0/5] check: add option to rerun failed tests

On Fri, Jul 08, 2022 at 10:51:37AM +0200, David Disseldorp wrote:
> This patchset adds support to loop on failed tests, as proposed by
> Ted Ts'o in https://lwn.net/Articles/897061/:
>   add a mode that will immediately rerun a failed test 25 or 100 times
>   to establish a failure percentage.

This patchset is good to me. if no more objections, I think it's time
to merge this feature. I'll give it a testing, then try to push it
this weekend.

Reviewed-by: Zorro Lang <zlang@...hat.com>

Thanks,
Zorro

> 
> Changes since (v3), following Darrick's review:
> - extended full/out.bad/etc. retention to cover more paths and remove
>   stale .rerun# files
> 
> Changes since (v2), following Darrick's review:
> - dropped RFC flag
> - rebased atop v2022.07.03
> - simplified test iterator
>   + results stashed at the end of each iteration, rather than start of
>     next / loop-exit
> - dropped aggregate loop stats message from xunit report
> - squashed full/dmesg/out.bad file retention patch with rerun patch
> 
> Changes since (v1):
> - rebased atop upstream v2022.06.26
> - added a few extra cleanup commits
> - append details for every rerun to xunit output
>   + provide aggregate stats via <testcase status=X>
> - extend _stash_test_status to call report hook, as well as save failure
>   artifacts with a .rerun# suffix
> 
> Diffstat:
>  check         | 137 +++++++++++++++++++++++++++++++++++++-------------
>  common/report |  90 ++++++++++++++++-----------------
>  2 files changed, 144 insertions(+), 83 deletions(-)
> 
>