Message-ID: <Zp4spjsaqQ85fVuk@elver.google.com>
Date: Mon, 22 Jul 2024 11:55:50 +0200
From: Marco Elver <elver@...gle.com>
To: Kees Cook <kees@...nel.org>
Cc: David Gow <davidgow@...gle.com>,
	Brendan Higgins <brendan.higgins@...ux.dev>,
	Rae Moar <rmoar@...gle.com>, John Hubbard <jhubbard@...dia.com>,
	Jonathan Corbet <corbet@....net>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
	linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-hardening@...r.kernel.org
Subject: Re: [PATCH v2] Documentation: KUnit: Update filename best practices

On Sat, Jul 20, 2024 at 09:54AM -0700, Kees Cook wrote:
> Based on feedback from Linus[1] and follow-up discussions, change the
> suggested file naming for KUnit tests.
> 
> Link: https://lore.kernel.org/lkml/CAHk-=wgim6pNiGTBMhP8Kd3tsB7_JTAuvNJ=XYd3wPvvk=OHog@mail.gmail.com/ [1]
> Signed-off-by: Kees Cook <kees@...nel.org>
[...]
>  Documentation/dev-tools/kunit/style.rst | 25 +++++++++++++++----------
>  1 file changed, 15 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/dev-tools/kunit/style.rst b/Documentation/dev-tools/kunit/style.rst
> index b6d0d7359f00..1538835cd0e2 100644
> --- a/Documentation/dev-tools/kunit/style.rst
> +++ b/Documentation/dev-tools/kunit/style.rst
> @@ -188,15 +188,20 @@ For example, a Kconfig entry might look like:
>  Test File and Module Names
>  ==========================
>  
> -KUnit tests can often be compiled as a module. These modules should be named
> -after the test suite, followed by ``_test``. If this is likely to conflict with
> -non-KUnit tests, the suffix ``_kunit`` can also be used.
> -
> -The easiest way of achieving this is to name the file containing the test suite
> -``<suite>_test.c`` (or, as above, ``<suite>_kunit.c``). This file should be
> -placed next to the code under test.
> +Whether a KUnit test is compiled as a separate module or via an
> +``#include`` in a core kernel source file, the file should be named
> +after the test suite, followed by ``_kunit``, and live in a ``tests``
> +subdirectory to avoid conflicting with regular modules (e.g. if "foobar"
> +is the core module, then "foobar_kunit" is the KUnit test module) or the
> +core kernel source file names (e.g. for tab-completion). Many existing
> +tests use a ``_test`` suffix, but this is considered deprecated.

What's wrong with the `_test` suffix (if inside a "tests" subdir)?

Rules are good, but please can we retain some common sense?

I understand the requirement for adding things to a "tests" subdir, so
that $foo.c is not right next to a $foo_test.c or $foo_kunit.c.

There are exception, however, if there is no $foo.c. For example:

	- mm/kfence/kfence_test.c
	- kernel/kcsan/kcsan_test.c
	- mm/kmsan/kmsan_test.c

In all these cases it'd be very annoying to move things into a "tests"
subdir, because there's only 1 test, and there isn't even a $foo.c file.
While there's a $foo.h file, I consider deeper directory nesting with 1
file in the subdir to be more annoying.

The rules should emphasize some basic guidelines, as they have until
now, and maybe add some additional suggestions to avoid the problem that
Linus mentioned. But _overfitting_ the new rules to avoid that single
problem is just adding more friction elsewhere if followed blindly.

> +So for the common case, name the file containing the test suite
> +``tests/<suite>_kunit.c``. The ``tests`` directory should be placed at
> +the same level as the code under test. For example, tests for
> +``lib/string.c`` live in ``lib/tests/string_kunit.c``.
>  
>  If the suite name contains some or all of the name of the test's parent
> -directory, it may make sense to modify the source filename to reduce redundancy.
> -For example, a ``foo_firmware`` suite could be in the ``foo/firmware_test.c``
> -file.
> +directory, it may make sense to modify the source filename to reduce
> +redundancy. For example, a ``foo_firmware`` suite could be in the
> +``tests/foo/firmware_kunit.c`` file.

I'm more confused now. This is just moving tests further away from what
they are testing for no good reason. If there's a directory "foo", then
moving things to "tests/foo" is unclear. It's unclear if "tests" is
inside parent of "foo" or actually a subdir of "foo". Per the paragraph
above, I inferred it's "foo/tests/foo/...", which is horrible. If it's
"../tests/foo/..." it's also bad because it's just moving tests further
away from what they are testing.

And keeping tests close to the source files under test is generally
considered good practice, as it avoids the friction required to discover
where tests live. Moving tests to "../tests" or "../../*/tests" in the
majority of cases is counterproductive.

It is more important for people to quickly discover tests nearby and
actually run them, vs. having them stashed away somewhere so they don't
bother us.

While we can apply common sense, all too often someone follows these
rules blindly and we end up with a mess.

Thanks,
- Marco

Powered by blists - more mailing lists