linux-kernel - Re: Converting kselftest test modules to kunit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CABVgOSkbOr28j7yD-M0Lk3G6sJHey_QjpGdLZWBise1Tbeumkg@mail.gmail.com>
Date: Tue, 30 Jul 2024 13:23:05 +0800
From: David Gow <davidgow@...gle.com>
To: Shuah Khan <skhan@...uxfoundation.org>
Cc: Muhammad Usama Anjum <usama.anjum@...labora.com>, Kees Cook <keescook@...omium.org>, 
	Shuah Khan <shuah@...nel.org>, 
	"open list : KERNEL SELFTEST FRAMEWORK" <linux-kselftest@...r.kernel.org>, open list <linux-kernel@...r.kernel.org>, 
	kunit-dev@...glegroups.com, "kernel@...labora.com" <kernel@...labora.com>
Subject: Re: Converting kselftest test modules to kunit

On Sat, 27 Jul 2024 at 03:35, Shuah Khan <skhan@...uxfoundation.org> wrote:
>
> On 7/15/24 04:09, Muhammad Usama Anjum wrote:
> > Hi Kees and All,
> >
> > There are several tests in kselftest subsystem which load modules to tests
> > the internals of the kernel. Most of these test modules are just loaded by
> > the kselftest, their status isn't read and reported to the user logs. Hence
> > they don't provide benefit of executing those tests.
> >
> > I've found patches from Kees where he has been converting such kselftests
> > to kunit tests [1]. The probable motivation is to move tests output of
> > kselftest subsystem which only triggers tests without correctly reporting
> > the results. On the other hand, kunit is there to test the kernel's
> > internal functions which can't be done by userspace.
> >
> > Kselftest:    Test user facing APIs from userspace
> > Kunit:                Test kernel's internal functions from kernelspace
> >
> > This brings me to conclusion that kselftest which are loading modules to
> > test kernelspace should be converted to kunit tests. I've noted several
> > such kselftests.
> >
> > This is just my understanding. Please mention if I'm correct above or more
> > reasons to support kselftest test modules transformation into kunit test.
> >
> > [1] https://lore.kernel.org/all/20221018082824.never.845-kees@kernel.org/
> >
>
> Please make sure you aren't taking away the ability to run these tests during
> boot. It doesn't make sense to convert every single test especially when it
> is intended to be run during boot without dependencies - not as a kunit test
> but a regression test during boot.

Given KUnit tests can run at boot (and, indeed, do by default if
enabled), I'd've assumed that this would be a good candidate for such
a conversion. It does add the KUnit 'dependency', but I can't think of
how that could be a problem. Is there a specific situation where
enabling CONFIG_KUNIT would cause problems?

> bitmap is one example - pay attention to the config help test - bitmap
> one clearly states it runs regression testing during boot. Any test that
> says that isn't a candidate for conversion.

Again, most KUnit tests are effectively regression tests at boot, so I
don't really understand what makes bitmap different. If it's just a
matter of there being a different interface to it, that's surely
something that we'll either be able to adapt to, or to have some
wrapper/shim to maintain compatibility. I agree that having needless
churn in formats is bad, but KUnit does seem the proper place for
these sorts of tests.

If this isn't the case, do we need to modify the testing guide to
mention this, as it definitely suggests KUnit for tests of in-kernel
functionality like this.

Cheers,
-- David

Download attachment "smime.p7s" of type "application/pkcs7-signature" (4014 bytes)