linux-kernel - Re: [PATCH] Documentation: kunit: Add CLI args for kunit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <8e0ad6ee-2542-1049-6106-b5dcc44ec164@riseup.net>
Date:   Tue, 19 Jul 2022 09:08:41 -0300
From:   Maíra Canal <mairacanal@...eup.net>
To:     Sadiya Kazi <sadiyakazi@...gle.com>, brendanhiggins@...gle.com,
        davidgow@...gle.com, skhan@...uxfoundation.org, corbet@....net,
        linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] Documentation: kunit: Add CLI args for kunit_tool

On 7/19/22 06:22, 'Sadiya Kazi' via KUnit Development wrote:
> Run_wrapper.rst was missing some command line arguments. Added
> additional args in the file. Included all initial review comments.
> 
> Signed-off-by: Sadiya Kazi <sadiyakazi@...gle.com>
> ---
>  Documentation/dev-tools/kunit/run_wrapper.rst | 48 +++++++++++++++++++
>  1 file changed, 48 insertions(+)
> 
> diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
> index 5e560f2c5fca..91f5dda36e83 100644
> --- a/Documentation/dev-tools/kunit/run_wrapper.rst
> +++ b/Documentation/dev-tools/kunit/run_wrapper.rst
> @@ -257,3 +257,51 @@ command line arguments:
>              added or modified. Instead, enable all tests
>              which have satisfied dependencies by adding
>              ``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
> +- ``--kunitconfig``: Specifies the path to the ``.kunitconfig`` file.
> +  This Kconfig fragment enables KUnit tests. The "/.kunitconfig" gets
> +  appended to the path specified. For example, If a directory path "lib/kunit"
You might want to change it for a lower-case "if", like: "For example,
if a directory path "lib/kunit"".

> +  is given, the complete path will be "lib/kunit/.kunitconfig".

This explanation may imply that if I pass `--kunitconfig
lib/kunit/.kunitconfig` it wouldn't work, as "/.kunitconfig" is always
appended to the path. But this is not true, both `--kunitconfig
lib/kunit/.kunitconfig` and `--kunitconfig lib/kunit` are valid options.

> +
> +- ``--kconfig_add``: Specifies additional configuration options to be
> +  appended to the ``.kunitconfig`` file. For example, ``CONFIG_KASAN=y``.

You could expand this example to:
```
./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y
```

> +
> +- ``--arch``: Runs tests with the specified architecture. The architecture

"Runs tests on the specified architecture."

> +  specified must match the string passed to the ARCH make parameter.

ARCH is not a "make parameter", ARCH is a environment variable.
Therefor, ARCH is not "passed to make", as it a environment variable on
the shell. Check the ARCH documentation [1].

> +  For example, i386, x86_64, arm, um, etc. Non-UML architectures run on QEMU.
> +  Default to 'um'.
> +
> +- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the
> +  same argument as passed to the ``CROSS_COMPILE`` variable used by> +  Kbuild. This will be the prefix for the toolchain
> +  binaries such as GCC. For example:
> +
> +  - ``sparc64-linux-gnu`` if we have the sparc toolchain installed on
> +    our system.
> +
> +  - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux``
> +    if we have downloaded the microblaze toolchain from the 0-day
> +    website to a specified path in our home directory called toolchains.

This is a copy of the explanation on [2]. You may want to delete the
previous explanation or change this one.

> +
> +- ``--qemu_config``: Specifies the path to the file containing a

"Specifies the path to a file containing a"

> +  custom qemu architecture definition. This should be a python file
> +  containing a QemuArchParams object.

"containing a `QemuArchParams` object."

The qemu_config has also a pretty good documentation on [2].

> +
> +- ``--qemu_args``: Specifies additional QEMU arguments, for example, "-smp 8".
> +
> +- ``--jobs``: Specifies the number of jobs (commands) to run simultaneously.
> +  By default, this is set to the number of cores on your system.

This CLI argument is already explained on [3]. You may want to delete
the previous explanation or change this one.

> +
> +- ``--timeout``: Specifies the maximum number of seconds allowed for all tests to run.
> +  This does not include the time taken to build the tests.

Same here.

> +
> +- ``--kernel_args``: Specifies the kernel command-line arguments. Might be repeated.
> +
> +- ``--run_isolated``: If set, boots the kernel for each individual suite/test.
> +  This is useful for debugging a non-hermetic test, one that
> +  might pass/fail based on what ran before it.
> +
> +- ``--raw_output``: If set, generates unformatted output from kernel.
> +  If set to ``--raw_output=kunit``, filters to just KUnit output.

This command line argument is already explain on the Parse Test Results
section [4].

> +
> +- ``--json``: If set, it stores the test results in a JSON format and prints to stdout or

"If set, it stores the test results in a JSON format and prints it to
`stdout` or"

> +  saves to a file if a filename is specified.

It's great to see new documentation for KUnit!

[1] https://docs.kernel.org/kbuild/kbuild.html#arch
[2]
https://docs.kernel.org/dev-tools/kunit/run_wrapper.html#run-tests-on-qemu
[3]
https://docs.kernel.org/dev-tools/kunit/run_wrapper.html#run-tests-with-kunit-tool
[4]
https://docs.kernel.org/dev-tools/kunit/run_wrapper.html#parse-test-results

Best Regards,
- Maíra Canal