linux-kernel - Re: [KTAP V2 PATCH] ktap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <7180be2d-60c7-0d99-338e-b818771b310f@gmail.com>
Date:   Sat, 11 Mar 2023 21:25:16 -0600
From:   Frank Rowand <frowand.list@...il.com>
To:     Rae Moar <rmoar@...gle.com>, davidgow@...gle.com,
        skhan@...uxfoundation.org, keescook@...omium.org,
        Tim.Bird@...y.com, brendanhiggins@...gle.com
Cc:     corbet@....net, guillaume.tucker@...labora.com,
        dlatypov@...gle.com, kernelci@...ups.io,
        kunit-dev@...glegroups.com, linux-kselftest@...r.kernel.org,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [KTAP V2 PATCH] ktap_v2: add skip test result

On 3/10/23 16:20, Rae Moar wrote:
> Add the test result "skip" to KTAP version 2 as an alternative way to
> indicate a test was skipped.
> 
> The current spec uses the "#SKIP" directive to indicate that a test was
> skipped. However, the "#SKIP" directive is not always evident when quickly
> skimming through KTAP results.
> 
> The "skip" result would provide an alternative that could make it clearer
> that a test has not successfully passed because it was skipped.
> 
> Before:
> 
>  KTAP version 1
>  1..1
>    KTAP version 1
>    1..2
>    ok 1 case_1
>    ok 2 case_2 #SKIP
>  ok 1 suite
> 
> After:
> 
>  KTAP version 2
>  1..1
>    KTAP version 2
>    1..2
>    ok 1 case_1
>    skip 2 case_2
>  ok 1 suite
> 
> Here is a link to a version of the KUnit parser that is able to parse
> the skip test result for KTAP version 2. Note this parser is still able
> to parse the "#SKIP" directive.
> 
> Link: https://kunit-review.googlesource.com/c/linux/+/5689
> 
> Signed-off-by: Rae Moar <rmoar@...gle.com>
> ---

< snip >

Another reason to add the "skip" result is that there was disagreement in previous
discussions as to whether the "#SKIP" directive should be used in an "ok" result
or a "not_ok" result.