Message-ID: <9cd80aa2-fc8d-1fed-838b-cf4951692b6d@infradead.org>
Date:   Mon, 23 Sep 2019 17:51:41 -0700
From:   Randy Dunlap <rdunlap@...radead.org>
To:     Brendan Higgins <brendanhiggins@...gle.com>,
        frowand.list@...il.com, gregkh@...uxfoundation.org,
        jpoimboe@...hat.com, keescook@...gle.com,
        kieran.bingham@...asonboard.com, mcgrof@...nel.org,
        peterz@...radead.org, robh@...nel.org, sboyd@...nel.org,
        shuah@...nel.org, tytso@....edu, yamada.masahiro@...ionext.com
Cc:     devicetree@...r.kernel.org, dri-devel@...ts.freedesktop.org,
        kunit-dev@...glegroups.com, linux-doc@...r.kernel.org,
        linux-fsdevel@...r.kernel.org, linux-kbuild@...r.kernel.org,
        linux-kernel@...r.kernel.org, linux-kselftest@...r.kernel.org,
        linux-nvdimm@...ts.01.org, linux-um@...ts.infradead.org,
        Alexander.Levin@...rosoft.com, Tim.Bird@...y.com,
        amir73il@...il.com, dan.carpenter@...cle.com, daniel@...ll.ch,
        jdike@...toit.com, joel@....id.au, julia.lawall@...6.fr,
        khilman@...libre.com, knut.omang@...cle.com, logang@...tatee.com,
        mpe@...erman.id.au, pmladek@...e.com, richard@....at,
        rientjes@...gle.com, rostedt@...dmis.org, wfg@...ux.intel.com,
        torvalds@...ux-foundation.org,
        Felix Guo <felixguoxiuping@...il.com>,
        Jonathan Corbet <corbet@....net>
Subject: Re: [PATCH v18 15/19] Documentation: kunit: add documentation for
 KUnit

On 9/23/19 2:02 AM, Brendan Higgins wrote:

> diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
> new file mode 100644
> index 000000000000..c6e69634e274
> --- /dev/null
> +++ b/Documentation/dev-tools/kunit/usage.rst
> @@ -0,0 +1,576 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +===========
> +Using KUnit
> +===========
> +
> +The purpose of this document is to describe what KUnit is, how it works, how it
> +is intended to be used, and all the concepts and terminology that are needed to
> +understand it. This guide assumes a working knowledge of the Linux kernel and
> +some basic knowledge of testing.
> +
> +For a high level introduction to KUnit, including setting up KUnit for your
> +project, see :doc:`start`.
> +
> +Organization of this document
> +=============================
> +
> +This document is organized into two main sections: Testing and Isolating
> +Behavior. The first covers what a unit test is and how to use KUnit to write

                              what unit tests are
would agree with the following "them."

> +them. The second covers how to use KUnit to isolate code and make it possible
> +to unit test code that was otherwise un-unit-testable.
> +
> +Testing
> +=======
> +

[snip]


> +
> +Test Suites
> +~~~~~~~~~~~
> +
> +Now obviously one unit test isn't very helpful; the power comes from having
> +many test cases covering all of your behaviors. Consequently it is common to

                   covering all of a unit's behaviors.

> +have many *similar* tests; in order to reduce duplication in these closely
> +related tests most unit testing frameworks provide the concept of a *test
> +suite*, in KUnit we call it a *test suite*; all it is is just a collection of

                                             . This is just a collection of

> +test cases for a unit of code with a set up function that gets invoked before
> +every test cases and then a tear down function that gets invoked after every

   every test case

> +test case completes.
> +
> +Example:
> +
> +.. code-block:: c
> +
> +	static struct kunit_case example_test_cases[] = {
> +		KUNIT_CASE(example_test_foo),
> +		KUNIT_CASE(example_test_bar),
> +		KUNIT_CASE(example_test_baz),
> +		{}
> +	};
> +
> +	static struct kunit_suite example_test_suite = {
> +		.name = "example",
> +		.init = example_test_init,
> +		.exit = example_test_exit,
> +		.test_cases = example_test_cases,
> +	};
> +	kunit_test_suite(example_test_suite);
> +
> +In the above example the test suite, ``example_test_suite``, would run the test
> +cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``,
> +each would have ``example_test_init`` called immediately before it and would
> +have ``example_test_exit`` called immediately after it.
> +``kunit_test_suite(example_test_suite)`` registers the test suite with the
> +KUnit test framework.
> +
> +.. note::
> +   A test case will only be run if it is associated with a test suite.
> +
> +For a more information on these types of things see the :doc:`api/test`.

   For more

> +
> +Isolating Behavior
> +==================
> +

[snip]

> +
> +.. _kunit-on-non-uml:
> +
> +KUnit on non-UML architectures
> +==============================
> +
> +By default KUnit uses UML as a way to provide dependencies for code under test.
> +Under most circumstances KUnit's usage of UML should be treated as an
> +implementation detail of how KUnit works under the hood. Nevertheless, there
> +are instances where being able to run architecture specific code, or test

                           I would drop the comma above.

> +against real hardware is desirable. For these reasons KUnit supports running on
> +other architectures.
> +
> +Running existing KUnit tests on non-UML architectures
> +-----------------------------------------------------
> +

[snip]

> +Writing new tests for other architectures
> +-----------------------------------------
> +
> +The first thing you must do is ask yourself whether it is necessary to write a
> +KUnit test for a specific architecture, and then whether it is necessary to
> +write that test for a particular piece of hardware. In general, writing a test
> +that depends on having access to a particular piece of hardware or software (not
> +included in the Linux source repo) should be avoided at all costs.
> +
> +Even if you only ever plan on running your KUnit test on your hardware
> +configuration, other people may want to run your tests and may not have access
> +to your hardware. If you write your test to run on UML, then anyone can run your
> +tests without knowing anything about your particular setup, and you can still
> +run your tests on your hardware setup just by compiling for your architecture.
> +
> +.. important::
> +   Always prefer tests that run on UML to tests that only run under a particular
> +   architecture, and always prefer tests that run under QEMU or another easy
> +   (and monitarily free) to obtain software environment to a specific piece of

           monetarily

> +   hardware.
> +
> +Nevertheless, there are still valid reasons to write an architecture or hardware
> +specific test: for example, you might want to test some code that really belongs
> +in ``arch/some-arch/*``. Even so, try your best to write the test so that it
> +does not depend on physical hardware: if some of your test cases don't need the
> +hardware, only require the hardware for tests that actually need it.
> +
> +Now that you have narrowed down exactly what bits are hardware specific, the
> +actual procedure for writing and running the tests is pretty much the same as
> +writing normal KUnit tests. One special caveat is that you have to reset
> +hardware state in between test cases; if this is not possible, you may only be
> +able to run one test case per invocation.
> +
> +.. TODO(brendanhiggins@...gle.com): Add an actual example of an architecture
> +   dependent KUnit test.


-- 
~Randy

Powered by blists - more mailing lists