linux-kernel - Re: [PATCH v3 1/3] kunit: Report the count of test suites in a module

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230801151711.2765e2eb@coco.lan>
Date:   Tue, 1 Aug 2023 15:17:11 +0200
From:   Mauro Carvalho Chehab <mchehab@...nel.org>
To:     Janusz Krzysztofik <janusz.krzysztofik@...ux.intel.com>
Cc:     Brendan Higgins <brendan.higgins@...ux.dev>,
        David Gow <davidgow@...gle.com>,
        linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
        igt-dev@...ts.freedesktop.org, intel-xe@...ts.freedesktop.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 1/3] kunit: Report the count of test suites in a
 module

Em Mon, 31 Jul 2023 16:10:23 +0200
Janusz Krzysztofik <janusz.krzysztofik@...ux.intel.com> escreveu:

> According to KTAP specification[1], results should always start from a
> header that provides a TAP protocol version, followed by a test plan with
> a count of items to be executed.  That pattern should be followed at each
> nesting level.  In the current implementation of the top-most, i.e., test
> suite level, those rules apply only for test suites built into the kernel,
> executed and reported on boot.  Results submitted to dmesg from kunit test
> modules loaded later are missing those top-level headers.
> 
> As a consequence, if a kunit test module provides more than one test suite
> then, without the top level test plan, external tools that are parsing
> dmesg for kunit test output are not able to tell how many test suites
> should be expected and whether to continue parsing after complete output
> from the first test suite is collected.
> 
> Submit the top-level headers also from the kunit test module notifier
> initialization callback.
> 
> [1] https://docs.kernel.org/dev-tools/ktap.html#
> 
> Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@...ux.intel.com>
> ---
>  lib/kunit/test.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> index 84e4666555c94..a29ca1acc4d81 100644
> --- a/lib/kunit/test.c
> +++ b/lib/kunit/test.c
> @@ -729,6 +729,11 @@ EXPORT_SYMBOL_GPL(__kunit_test_suites_exit);
>  #ifdef CONFIG_MODULES
>  static void kunit_module_init(struct module *mod)
>  {
> +	if (mod->num_kunit_suites > 0) {
> +		pr_info("KTAP version 1\n");
> +		pr_info("1..%d\n", mod->num_kunit_suites);
> +	}
> +
>  	__kunit_test_suites_init(mod->kunit_suites, mod->num_kunit_suites);
>  }

IMO, the best would be instead to export kunit_exec_run_tests() and
use it here too.

Except for the nit, LGTM.


Thanks,
Mauro