linux-kernel - Re: [PATCH v5 linux-kselftest-test 3/6] kunit: allow kunit tests to be loaded as a module

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LRH.2.20.1912041531160.5511@dhcp-10-175-179-22.vpn.oracle.com>
Date:   Wed, 4 Dec 2019 15:41:28 +0000 (GMT)
From:   Alan Maguire <alan.maguire@...cle.com>
To:     "Theodore Y. Ts'o" <tytso@....edu>
cc:     Brendan Higgins <brendanhiggins@...gle.com>,
        Alan Maguire <alan.maguire@...cle.com>,
        Iurii Zaikin <yzaikin@...gle.com>,
        David Gow <davidgow@...gle.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        KUnit Development <kunit-dev@...glegroups.com>,
        Kees Cook <keescook@...omium.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        catalin.marinas@....com, joe.lawrence@...hat.com,
        penguin-kernel@...ove.sakura.ne.jp, urezki@...il.com,
        andriy.shevchenko@...ux.intel.com,
        Jonathan Corbet <corbet@....net>, adilger.kernel@...ger.ca,
        Luis Chamberlain <mcgrof@...nel.org>,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        Knut Omang <knut.omang@...cle.com>
Subject: Re: [PATCH v5 linux-kselftest-test 3/6] kunit: allow kunit tests to
 be loaded as a module

On Tue, 3 Dec 2019, Theodore Y. Ts'o wrote:

> On Tue, Dec 03, 2019 at 09:54:25AM -0800, Brendan Higgins wrote:
> > On Tue, Dec 3, 2019 at 4:08 AM Alan Maguire <alan.maguire@...cle.com> wrote:
> > >
> > > As tests are added to kunit, it will become less feasible to execute
> > > all built tests together.  By supporting modular tests we provide
> > > a simple way to do selective execution on a running system; specifying
> > >
> > > CONFIG_KUNIT=y
> > > CONFIG_KUNIT_EXAMPLE_TEST=m
> > >
> > > ...means we can simply "insmod example-test.ko" to run the tests.
> > >
> > > To achieve this we need to do the following:
> > >
> > > o export the required symbols in kunit
> > > o string-stream tests utilize non-exported symbols so for now we skip
> > >   building them when CONFIG_KUNIT_TEST=m.
> > > o support a new way of declaring test suites.  Because a module cannot
> > >   do multiple late_initcall()s, we provide a kunit_test_suites() macro
> > >   to declare multiple suites within the same module at once.
> > > o some test module names would have been too general ("test-test"
> > >   and "example-test" for kunit tests, "inode-test" for ext4 tests);
> > >   rename these as appropriate ("kunit-test", "kunit-example-test"
> > >   and "ext4-inode-test" respectively).
> > >
> > > Co-developed-by: Knut Omang <knut.omang@...cle.com>
> > > Signed-off-by: Knut Omang <knut.omang@...cle.com>
> > > Signed-off-by: Alan Maguire <alan.maguire@...cle.com>
> > 
> > Reviewed-by: Brendan Higgins <brendanhiggins@...gle.com>
> 
> Acked-by: Theodore Ts'o <tytso@....edu> # for ext4 bits
> 

Thanks for taking a look!

> 
> I do have one question, out of curiosity --- for people who aren't
> using UML to run Kunit tests, and are either running the kunit tests
> during boot, or when the module is loaded, is there the test framework
> to automatically extract the test reports out of dmesg?
> 
> I can boot a kernel with kunit tests enabled using kvm, and I see it
> splatted intermixed with the rest of the kernel boot messages.  This
> is how I tested the 32-bit ext4 inode test fix.  But I had to manually
> find the test output.  Is that the expected way people are supposed to
> be using Kunit tests w/o using UML and the python runner?
>

Looks like Brendan's got something coming to resolve this;
I've also got a patch that I was hoping to send out soon
that might help.  The idea is that each test suite would create
a debugfs representation under /sys/kernel/debug/kunit;
specifically:

/sys/kernel/debug/kunit/results/<suite>
/sys/kernel/debug/kunit/results/<suite>-tests

...where cat'ing the former shows the full set of results,
and the latter is a directory within which we can display
individual test results in test-case-specific files.

This is all done by ensuring that when tests log information,
they log to a per-test-case log buffer as well as to dmesg.

If the above sounds useful, I'll try and polish up the patch
for submission. Thanks!

Alan

> Thanks,
> 
> 						- Ted
>