linux-kernel - Re: [PATCH] Documentation: kunit: Clarify test filter format

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGS_qxr_vdix2mnoZve+L4EZv4wO-XdnJ1burnn7fM-at6EuEg@mail.gmail.com>
Date: Thu, 28 Mar 2024 11:27:01 -0700
From: Daniel Latypov <dlatypov@...gle.com>
To: Brendan Jackman <jackmanb@...gle.com>
Cc: linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com, 
	linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org, 
	Brendan Higgins <brendan.higgins@...ux.dev>, davidgow@...gle.com, rmoar@...gle.com, 
	corbet@....net
Subject: Re: [PATCH] Documentation: kunit: Clarify test filter format

On Thu, Mar 28, 2024 at 7:20 AM 'Brendan Jackman' via KUnit
Development <kunit-dev@...glegroups.com> wrote:
>
> It seems obvious once you know, but at first I didn't realise that the
> suite name is part of this format. Document it and add example.
>
> Signed-off-by: Brendan Jackman <jackmanb@...gle.com>
> ---
>  Documentation/dev-tools/kunit/run_wrapper.rst | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
> index 19ddf5e07013..e75a5fc05814 100644
> --- a/Documentation/dev-tools/kunit/run_wrapper.rst
> +++ b/Documentation/dev-tools/kunit/run_wrapper.rst
> @@ -156,13 +156,20 @@ Filtering tests
>  ===============
>
>  By passing a bash style glob filter to the ``exec`` or ``run``
> -commands, we can run a subset of the tests built into a kernel . For
> +commands, we can run a subset of the tests built into a kernel,
> +identified by a string like ``$suite_name.$test_name``. For

Apologies for the overly terse docs, that's my fault :)
I'm wondering if we can further improve it while we're here.

Note, the format for the glob is: $suite_name[.$test_name].

This current wording and examples (before and after this change) might
make the user think otherwise, i.e. that it works like
  effective_name = suite_name + '.' + test_name
  return glob_matches(effective_name, filter_glob)

E.g. given a test name like `suite.test_name` and glob='suite*name'
they might expect it to match, but it does *not*.

The logic actually works like:
  suite_glob, test_glob = split(filter_glob)
  if not_glob_matches(suite_name, suite_glob):
     return False
  if test_glob and not glob_matches(test_name, test_glob):
     return False
  return True

Perhaps expanding the list of examples to cover more of the edge cases
could help get the right intuition?

E.g. perhaps these:
  kunit.py run <suite_name>  # runs all tests in a specific suite
  kunit.py run <suite_name>.<test_name>  # run a specific test

  kunit.py run suite_prefix*  # what the current example shows
  kunit.py run *.*test_suffix  # matches all suites, only tests w/ a
certain suffix
  kunit.py run suite_prefix*.*test_suffix # combined version of above

Thoughts?

Thanks,
Daniel