linux-kernel - Re: Converting kselftest test modules to kunit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <202407161005.CACE2E355@keescook>
Date: Tue, 16 Jul 2024 10:59:06 -0700
From: Kees Cook <kees@...nel.org>
To: Muhammad Usama Anjum <usama.anjum@...labora.com>
Cc: Shuah Khan <shuah@...nel.org>, davidgow@...gle.com,
	"open list : KERNEL SELFTEST FRAMEWORK" <linux-kselftest@...r.kernel.org>,
	open list <linux-kernel@...r.kernel.org>,
	kunit-dev@...glegroups.com,
	"kernel@...labora.com" <kernel@...labora.com>
Subject: Re: Converting kselftest test modules to kunit

On Tue, Jul 16, 2024 at 01:11:14PM +0500, Muhammad Usama Anjum wrote:
> On 7/15/24 9:40 PM, Kees Cook wrote:
> > On Mon, Jul 15, 2024 at 03:09:24PM +0500, Muhammad Usama Anjum wrote:
> >> Hi Kees and All,
> >>
> >> There are several tests in kselftest subsystem which load modules to tests
> >> the internals of the kernel. Most of these test modules are just loaded by
> >> the kselftest, their status isn't read and reported to the user logs. Hence
> >> they don't provide benefit of executing those tests.
> >>
> >> I've found patches from Kees where he has been converting such kselftests
> >> to kunit tests [1]. The probable motivation is to move tests output of
> >> kselftest subsystem which only triggers tests without correctly reporting
> >> the results. On the other hand, kunit is there to test the kernel's
> >> internal functions which can't be done by userspace.
> >>
> >> Kselftest:	Test user facing APIs from userspace
> >> Kunit:		Test kernel's internal functions from kernelspace
> > 
> > I would say this is a reasonable guide to how these things should
> > be separated, yes. That said, much of what was kind of ad-hoc kernel
> > internals testing that was triggered via kselftests is better done via
> > KUnit these days, but not everything.
> I started investigated when I found that kselftest doesn't parse the kernel
> logs to mark these tests pass/fail. (kselftest/lib is good example of it)
> 
> > 
> >> This brings me to conclusion that kselftest which are loading modules to
> >> test kernelspace should be converted to kunit tests. I've noted several
> >> such kselftests.
> > 
> > I would tend to agree, yes. Which stand out to you? I've mainly been
> > doing the conversions when I find myself wanting to add new tests, etc.
> lib
> 	test_bitmap
> 	prime_numbers
> 	test_printf
> 	test_scanf

Yeah, these would be nice to convert.

> 	test_strscpy (already converted, need to remove this test)

Yup, converted in bb8d9b742aa7 ("string: Merge strscpy KUnit tests into string_kunit.c")

> lock
> 	test-ww_mutex module
> net
> 	test_blackhole_dev

I don't know these very well, but yeah worth looking into.

> user
> 	test_user_copy (probably already converted, need to remove this test)

This is done in -next via cf6219ee889f ("usercopy: Convert test_user_copy to KUnit test")

> firmware
> 	test_firmware

This might not work to convert: there's a userspace half for testing
firmware loading (see the kselftest side...)

> fpu
> 	test_fpu

Seems reasonable.

> Most of these modules are found in lib/*.
> 
> Would it be desired to move these to kunit?

Checking with the authors/maintainer is probably the first thing to do;
check the git history to see who has been working on them.

-- 
Kees Cook