linux-kernel - Re: [PATCH v2 2/2] kunit: improve KTAP compliance of KUnit test output

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 21 Nov 2022 12:32:21 -0800
From:   Daniel Latypov <dlatypov@...gle.com>
To:     Rae Moar <rmoar@...gle.com>
Cc:     brendanhiggins@...gle.com, davidgow@...gle.com,
        skhan@...uxfoundation.org, mauro.chehab@...ux.intel.com,
        kunit-dev@...glegroups.com, linux-kernel@...r.kernel.org,
        linux-kselftest@...r.kernel.org, isabbasso@...eup.net,
        anders.roxell@...aro.org
Subject: Re: [PATCH v2 2/2] kunit: improve KTAP compliance of KUnit test output

On Mon, Nov 21, 2022 at 10:48 AM Rae Moar <rmoar@...gle.com> wrote:
>
> Change KUnit test output to better comply with KTAP v1 specifications
> found here: https://kernel.org/doc/html/latest/dev-tools/ktap.html.
> 1) Use "KTAP version 1" instead of "TAP version 14" as test output header
> 2) Remove '-' between test number and test name on test result lines
> 2) Add KTAP version lines to each subtest header as well
>
> Note that the new KUnit output still includes the “# Subtest” line now
> located after the KTAP version line. This does not completely match the
> KTAP v1 spec but since it is classified as a diagnostic line, it is not
> expected to be disruptive or break any existing parsers. This
> “# Subtest” line comes from the TAP 14 spec
> (https://testanything.org/tap-version-14-specification.html)
> and it is used to define the test name before the results.
>
> Original output:
>
>  TAP version 14
>  1..1
>    # Subtest: kunit-test-suite
>    1..3
>    ok 1 - kunit_test_1
>    ok 2 - kunit_test_2
>    ok 3 - kunit_test_3
>  # kunit-test-suite: pass:3 fail:0 skip:0 total:3
>  # Totals: pass:3 fail:0 skip:0 total:3
>  ok 1 - kunit-test-suite
>
> New output:
>
>  KTAP version 1
>  1..1
>    KTAP version 1
>    # Subtest: kunit-test-suite
>    1..3
>    ok 1 kunit_test_1
>    ok 2 kunit_test_2
>    ok 3 kunit_test_3
>  # kunit-test-suite: pass:3 fail:0 skip:0 total:3
>  # Totals: pass:3 fail:0 skip:0 total:3
>  ok 1 kunit-test-suite
>
> Signed-off-by: Rae Moar <rmoar@...gle.com>
> Reviewed-by: Daniel Latypov <dlatypov@...gle.com>
> Reviewed-by: David Gow <davidgow@...gle.com>
> ---
>
> Changes since v1:
> https://lore.kernel.org/all/20221104194705.3245738-1-rmoar@google.com/
> - Switch order of patches to make changes to the parser before making
>   changes to the test output
> - Change location of the new KTAP version line in subtest header to be
>   before the subtest header line

This patch still looks good to me. In fact, it looks better.

I prefer this updated version since this works a bit better with
debugfs. This way, kunit.py won't just skip over the subtest line when
looking for the initial KTAP header.

Daniel