[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <a248b7b5-6d3b-8bc9-a36e-58a311bb27b8@riseup.net>
Date: Fri, 19 Aug 2022 08:22:20 -0300
From: Maíra Canal <mairacanal@...eup.net>
To: Tales Aparecida <tales.aparecida@...il.com>,
Sadiya Kazi <sadiyakazi@...gle.com>
Cc: linux-kernel@...r.kernel.org, kunit-dev@...glegroups.com,
linux-doc@...r.kernel.org, linux-kselftest@...r.kernel.org,
davidgow@...gle.com, corbet@....net, brendan.higgins@...ux.dev,
Trevor Woerner <twoerner@...il.com>, siqueirajordao@...eup.net,
mwen@...lia.com, andrealmeid@...eup.net,
Isabella Basso <isabbasso@...eup.net>, magalilemes00@...il.com
Subject: Re: [PATCH 4/8] Documentation: KUnit: Reword start guide for
selecting tests
Hi Tales
On 8/19/22 02:32, Tales Aparecida wrote:
> Reword "Creating a ``.kunitconfig``" into "Selecting which tests to run"
> covering the current alternatives for editing configs and glob-filtering
>
> Signed-off-by: Tales Aparecida <tales.aparecida@...il.com>
> ---
> Documentation/dev-tools/kunit/start.rst | 90 +++++++++++++++++--------
> 1 file changed, 63 insertions(+), 27 deletions(-)
>
> diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
> index e4b73adde6d0..f0ec64207bd3 100644
> --- a/Documentation/dev-tools/kunit/start.rst
> +++ b/Documentation/dev-tools/kunit/start.rst
> @@ -52,27 +52,20 @@ The tests will pass or fail.
> For detailed information on this wrapper, see:
> Documentation/dev-tools/kunit/run_wrapper.rst.
>
> -Creating a ``.kunitconfig``
> ----------------------------
> -
> -By default, kunit_tool runs a selection of tests. However, you can specify which
> -unit tests to run by creating a ``.kunitconfig`` file with kernel config options
> -that enable only a specific set of tests and their dependencies.
> -The ``.kunitconfig`` file contains a list of kconfig options which are required
> -to run the desired targets. The ``.kunitconfig`` also contains any other test
> -specific config options, such as test dependencies. For example: the
> -``FAT_FS`` tests - ``FAT_KUNIT_TEST``, depends on
> -``FAT_FS``. ``FAT_FS`` can be enabled by selecting either ``MSDOS_FS``
> -or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
> +Selecting which tests to run
> +----------------------------
>
> -.. code-block:: none
> +By default, kunit_tool runs all tests reachable with minimal configuration,
> +that is, using default values for most of the kconfig options. However,
> +you can select which tests to run by:
>
> - CONFIG_KUNIT=y
> - CONFIG_MSDOS_FS=y
> - CONFIG_FAT_KUNIT_TEST=y
> +- `Customizing Kconfig`_ used to compile the kernel, or
> +- `Filtering tests by name`_ to select specifically which compiled tests to run.
>
> -1. A good starting point for the ``.kunitconfig`` is the KUnit default config.
> - You can generate it by running:
> +Customizing Kconfig
> +~~~~~~~~~~~~~~~~~~~
> +A good starting point for the ``.kunitconfig`` is the KUnit default config.
> +If you didn't run ``kunit.py run`` yet, you can generate it by running:
>
> .. code-block:: bash
>
> @@ -84,27 +77,70 @@ or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has:
> ``.kunitconfig`` lives in the ``--build_dir`` used by kunit.py, which is
> ``.kunit`` by default.
>
> -.. note ::
> +Before running the tests, kunit_tool ensures that all config options
> +set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
> +you if you have not included dependencies for the options used.
> +
> +There are many ways to customize the configurations:
> +
> +a. Edit ``.kunit/.kunitconfig``. The file should contain the list of kconfig
> + options required to run the desired tests, including their dependencies.
> You may want to remove CONFIG_KUNIT_ALL_TESTS from the ``.kunitconfig`` as
> it will enable a number of additional tests that you may not want.
> + If you need to run on an architecture other than UML see :ref:`kunit-on-qemu`.
>
> -2. You can then add any other Kconfig options, for example:
> +b. Enable additional kconfig options on top of ``.kunit/.kunitconfig``.
> + For example, to include the kernel's linked-list test you can run::
>
> -.. code-block:: none
> + ./tools/testing/kunit/kunit.py run \
> + --kconfig_add CONFIG_LIST_KUNIT_TEST=y
>
> - CONFIG_LIST_KUNIT_TEST=y
> +c. Provide the path of one or more .kunitconfig files from the tree.
> + For example, to run only ``FAT_FS`` and ``EXT4`` tests you can run::
>
> -Before running the tests, kunit_tool ensures that all config options
> -set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn
> -you if you have not included dependencies for the options used.
> + ./tools/testing/kunit/kunit.py run \
> + --kunitconfig ./fs/fat/.kunitconfig \
> + --kunitconfig ./fs/ext4/.kunitconfig
>
> -.. note ::
> - If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
> +d. If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the
> ``.config`` file. But you can edit the ``.config`` file directly or with
> tools like ``make menuconfig O=.kunit``. As long as its a superset of
> ``.kunitconfig``, kunit.py won't overwrite your changes.
>
>
> +.. note ::
> +
> + To save a .kunitconfig after finding a satisfactory configuration::
> +
> + make savedefconfig O=.kunit
> + cp .kunit/defconfig .kunit/.kunitconfig
> +
> +Filtering tests by name
> +~~~~~~~~~~~~~~~~~~~~~~~
> +If you want to be more specific than Kconfig can provide, it is also possible
> +to select which tests to execute at boot-time by passing a glob filter
> +(read instructions regarding the pattern in the manpage :manpage:`glob(7)`).
> +If there is a ``"."`` (period) in the filter, it will be interpreted as a
> +separator between the name of the test-suite and the test-case,
> +otherwise, it will be interpreted as the name of the test suite.
> +For example, let's assume we are using the default config:
Another small nit: you could keep the consistency on naming test-suite
and test-cases. Currently, you are using "test-suite" and "test suite"
(same for test-cases). I guess any of the two options are fine, but it
would be nice to keep it consistent.
> +
> +a. inform the name of a test suite, like ``"kunit_executor_test"``,
> + to run every test case it contains::
> +
> + ./tools/testing/kunit/kunit.py run "kunit_executor_test"
> +
> +b. inform the name of a test case prefixed by its test suite,
> + like ``"example.example_simple_test"``, to run specifically that test case::
> +
> + ./tools/testing/kunit/kunit.py run "example.example_simple_test"
> +
> +c. use wildcard characters (``*?[``) to run any test-case that match the pattern,
I guess it would be "matches" instead of "match" here.
Other than this small nits, this is a great improvement on the docs!
Reviewed-by: Maíra Canal <mairacanal@...eup.net>
Best Regards,
- Maíra Canal
> + like ``"*.*64*"`` to run test-cases containing ``"64"`` in the name inside
> + any test-suite::
> +
> + ./tools/testing/kunit/kunit.py run "*.*64*"
> +
> Running Tests without the KUnit Wrapper
> =======================================
> If you do not want to use the KUnit Wrapper (for example: you want code
Powered by blists - more mailing lists