Message-ID: <20250812094107.GA27450@lst.de>
Date: Tue, 12 Aug 2025 11:41:07 +0200
From: Christoph Hellwig <hch@....de>
To: Thomas Weißschuh <thomas.weissschuh@...utronix.de>
Cc: Christoph Hellwig <hch@....de>, Shuah Khan <shuah@...nel.org>,
	Masahiro Yamada <masahiroy@...nel.org>,
	Nathan Chancellor <nathan@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>, Willy Tarreau <w@....eu>,
	Thomas Weißschuh <linux@...ssschuh.net>,
	Brendan Higgins <brendan.higgins@...ux.dev>,
	David Gow <davidgow@...gle.com>, Rae Moar <rmoar@...gle.com>,
	Jonathan Corbet <corbet@....net>,
	Nicolas Schier <nicolas.schier@...ux.dev>,
	Kees Cook <kees@...nel.org>,
	Alexander Viro <viro@...iv.linux.org.uk>,
	Christian Brauner <brauner@...nel.org>, Jan Kara <jack@...e.cz>,
	Luis Chamberlain <mcgrof@...nel.org>,
	Christophe Leroy <christophe.leroy@...roup.eu>,
	linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
	linux-doc@...r.kernel.org, workflows@...r.kernel.org,
	linux-mm@...ck.org, linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH v5 00/15] kunit: Introduce UAPI testing framework

On Mon, Aug 04, 2025 at 05:01:35PM +0200, Thomas Weißschuh wrote:
> > That's great.  Let's reuse it without having to drive running userspace
> > programs from kernel code.
> 
> Running in the kernel is the point behind KUnit.

When using kunit as is to unit test kernel functionality - obviously.

When running it to integration test the syscall boundary - not at all.

> It could be done by putting
> all the userspace test into a initramfs and run them on boot from there.
> But that has other drawbacks:
> * The tests can't be run on an existing system.
> * All tests need to be loaded into memory together, and not on demand.
> * The tests can not be rerun.

None of that is true.  While running syscall level tests from an
initramfs could be a nice feature for an automatd kernel CI system,
nothin in this tests should require running from an initramfs.

> This was a response to one specific statement. Could you be a bit more specific
> in your critique? I am not sure what exactly you mean in some cases, making it
> hard to respond properly. For example "bloat", it is bloaty
> * source code,
> * object code for users enabling the new kconfig options,
> * object code for other users *not* enabling the new kconfig options?

You are adding kernel code both at the source and object level to run
userspace tests.  That is very clearly bloat.  Even more so as it adds
functionality and exports that don't fit in with what the kernel already
does for actual kernel functionality.

> > > It is not and most probably won't ever be. The maintainers of each testcase
> > > will decide which libc to use. Like it is in tools/testing/selftests/ today.
> > > Some use glibc, some nolibc and some can do both.
> > 
> > So why do you want to use it here?  And how is is related to the rest
> > of the series?
> 
> To make it easier to test a wide range of architectures by not requiring a
> libc from the toolchain. It also avoids relying on a bunch of out-of-tree
> code (glibc) as part of the test. And there are existing kselftests which
> use it over glibc for their own reasons.
> 
> But using nolibc in test code is not necessary and nobody is forced to do so.
> 
> (Maybe a disclaimer that I'm one of the nolibc maintainers is in order)

Well, why do you even mix it up with this unrelated series then?

> To run kselftests we need the following things:
> a) A toolchain which can build userspace executables.

You'll need that for any userspace program, no matter what test
harness.

> b) Quite a bit of supporting userland, at least glibc, coreutils and bash.

Well, a libc you will need anyway.  Maybe nolibc is good enough for
some tests, but as you already stated not for very many.  The others
are just an artifcat of how you run tests.

> c) A rootfs assembled out of these.
> d) An efficient way to incrementally rebuild the test executables and rootfs.
> e) A way to put that rootfs into the system under test.

You don't need a rootfs.

> And for all of this there should be good in-tree tooling.

Absolutely.

> Moving to a pure userspace solution would preclude the usage of KUnit as far as
> I can see.

You've still failed why using kunit is the goal and not just something
that made your life easier archieving your goal.

> > Yes, kselftests suck as most people will agree. But the answer is not
> > to add a lot of kernel bloat to treat userspace integration tests
> > like kernel units tests.
> 
> I fail to understand how this test code is worse than the existing KUnit test
> code. This is not meant to test complex scenarios, but single system calls or
> specific UAPIs, which may depend on architecture features. For example timers,
> signals, vDSO, mm etc.

So now having another half-assed test framework is a good thing?

> > How about you just fix kselftests, preferably
> > by reusing well known and teststed userland code?
> 
> Is "well known and tested userland code" referring to glibc or testing
> frameworks? As mentioned above, glibc can be used just fine and the frameworks
> I know about are lacking.

Basically any kind of testing framework that has broad use.  It's not
like there aren't userland unit test frameworks if you really want to
use that.

Powered by blists - more mailing lists