linux-kernel - Re: [PATCH] kunit: Reset suite count after running tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAGS_qxqFULDPy65kk9=H23RJk+BWr+dsUD9fBnm3pQn2zU9KcQ@mail.gmail.com>
Date:   Wed, 6 Oct 2021 12:53:29 -0700
From:   Daniel Latypov <dlatypov@...gle.com>
To:     David Gow <davidgow@...gle.com>
Cc:     Brendan Higgins <brendanhiggins@...gle.com>,
        Shuah Khan <skhan@...uxfoundation.org>,
        kunit-dev@...glegroups.com, linux-kselftest@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] kunit: Reset suite count after running tests

On Tue, Oct 5, 2021 at 9:41 PM David Gow <davidgow@...gle.com> wrote:
>
> There are some KUnit tests (KFENCE, Thunderbolt) which, for various
> reasons, do not use the kunit_test_suite() macro and end up running
> before the KUnit executor runs its tests. This means that their results
> are printed separately, and they aren't included in the suite count used
> by the executor.
>
> This causes the executor output to be invalid TAP, however, as the suite
> numbers used are no-longer 1-based, and don't match the test plan.
> kunit_tool, therefore, prints a large number of warnings.
>
> While it'd be nice to fix the tests to run in the executor, in the
> meantime, reset the suite counter to 1 in __kunit_test_suites_exit.
> Not only does this fix the executor, it means that if there are multiple
> calls to __kunit_test_suites_init() across different tests, they'll each
> get their own numbering.
>
> kunit_tool likes this better: even if it's lacking the results for those
> tests which don't use the executor (due to the lack of TAP header), the
> output for the other tests is valid.
>
> Signed-off-by: David Gow <davidgow@...gle.com>

Reviewed-by: Daniel Latypov <dlatypov@...gle.com>

I had been hoping we could pass in a kunit_context object of sorts to
__kunit_test_suites_init() and exit.
That feels like that would be the right place to store this counter in
the future.

But that's currently blocked on the aforementioned tests going outside
the executor and using these funcs directly.
So for now, this seems like the simplest and most sensible fix.

I wonder if we should add a TODO about the counter...
But eh, it was already a static variable before, so this isn't really
increasing the hackiness too much.

> ---
>  lib/kunit/test.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> index f246b847024e..3bd741e50a2d 100644
> --- a/lib/kunit/test.c
> +++ b/lib/kunit/test.c
> @@ -190,10 +190,10 @@ enum kunit_status kunit_suite_has_succeeded(struct kunit_suite *suite)
>  }
>  EXPORT_SYMBOL_GPL(kunit_suite_has_succeeded);
>
> +static size_t kunit_suite_counter = 1;
> +
>  static void kunit_print_subtest_end(struct kunit_suite *suite)
>  {
> -       static size_t kunit_suite_counter = 1;
> -
>         kunit_print_ok_not_ok((void *)suite, false,
>                               kunit_suite_has_succeeded(suite),
>                               kunit_suite_counter++,
> @@ -583,6 +583,8 @@ void __kunit_test_suites_exit(struct kunit_suite **suites)
>
>         for (i = 0; suites[i] != NULL; i++)
>                 kunit_exit_suite(suites[i]);
> +
> +       kunit_suite_counter = 1;
>  }
>  EXPORT_SYMBOL_GPL(__kunit_test_suites_exit);
>
> --
> 2.33.0.800.g4c38ced690-goog
>