linux-kernel - Re: [PATCH v5 00/15] kunit: Introduce UAPI testing framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250717132259.GA25835@lst.de>
Date: Thu, 17 Jul 2025 15:23:00 +0200
From: Christoph Hellwig <hch@....de>
To: Thomas Weißschuh <thomas.weissschuh@...utronix.de>
Cc: Masahiro Yamada <masahiroy@...nel.org>,
	Nathan Chancellor <nathan@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>, Willy Tarreau <w@....eu>,
	Thomas Weißschuh <linux@...ssschuh.net>,
	Brendan Higgins <brendan.higgins@...ux.dev>,
	David Gow <davidgow@...gle.com>, Rae Moar <rmoar@...gle.com>,
	Shuah Khan <shuah@...nel.org>, Jonathan Corbet <corbet@....net>,
	Nicolas Schier <nicolas.schier@...ux.dev>,
	Kees Cook <kees@...nel.org>,
	Alexander Viro <viro@...iv.linux.org.uk>,
	Christian Brauner <brauner@...nel.org>, Jan Kara <jack@...e.cz>,
	Christoph Hellwig <hch@....de>,
	Luis Chamberlain <mcgrof@...nel.org>,
	Christophe Leroy <christophe.leroy@...roup.eu>,
	linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
	linux-doc@...r.kernel.org, workflows@...r.kernel.org,
	linux-mm@...ck.org, linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH v5 00/15] kunit: Introduce UAPI testing framework

On Thu, Jul 17, 2025 at 10:48:02AM +0200, Thomas Weißschuh wrote:
> Currently testing of userspace and in-kernel API use two different
> frameworks.

Which is kinda expected as one has to run in the kernel to test
in-kernel kernel space APIs, and the other tests externally provided
kernel functionality.

> Therefore kunit is much easier to run against different kernel
> configurations and architectures.

Which is is normal.  unit tests are always easier to run than
integration tests.

> This series aims to combine kselftests and kunit, avoiding both their
> limitations. It works by compiling the userspace kselftests as part of
> the regular kernel build, embedding them into the kunit kernel or module
> and executing them from there.

This is really weird.  "Running userspace code is hard, so we package
it in the kernel".  I had my own fair share of problems with kselftests,
mostly because of the lack of structure and automated way to run them,
but adding them to the kernel (or a module) is overshooting the target
by far.

> If the kernel toolchain is not fit to
> produce userspace because of a missing libc, the kernel's own nolibc can
> be used instead.

Is nolibc enough to run all the selftests?  If so we should just do
it unconditionally, but linking to different libraries by availability
seems a bit problematic.

> The structured TAP output from the kselftest is integrated into the
> kunit KTAP output transparently, the kunit parser can parse the combined
> logs together.

Good idea!