linux-kernel - Re: [PATCH 1/3] selftests/timens: Print TAP headers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250509172639-61bade20-67f9-4815-8316-1bb16749f8d9@linutronix.de>
Date: Fri, 9 May 2025 17:41:03 +0200
From: Thomas Weißschuh <thomas.weissschuh@...utronix.de>
To: Shuah Khan <skhan@...uxfoundation.org>
Cc: Shuah Khan <shuah@...nel.org>, "Bird, Timothy" <Tim.Bird@...y.com>, 
	linux-kselftest@...r.kernel.org, linux-kernel@...r.kernel.org, Kees Cook <kees@...nel.org>
Subject: Re: [PATCH 1/3] selftests/timens: Print TAP headers

+Cc Kees

On Wed, May 07, 2025 at 03:06:16PM -0600, Shuah Khan wrote:
> On 5/2/25 06:03, Thomas Weißschuh wrote:
> > The TAP specification requires that the output begins with a header line.
> > These headers lines are missing in the timens tests.
> > 
> > Print such a line.
> 
> There is no cover letter for this - so I will respond to the first
> patch.

Hm, I sent one and can also see it on lore.

> The TAP information is added by the kselftest wrapper if
> you were to run the test using ksefltest.
> 
> The following will add the TAP header or if you use make kselftest
> command from the main Makefile.
> 
> make -C timens run_tests
> 
> cd timens; make run_tests (will also add TAP header)
> 
> The only time you won't see the TAP headers is when you run the test
> from the test directory just as a command. Is this what you need
> to do? I would rather not see TAP headers added to invidual tests
> unless there is a good reason for it.

Yes, I am running each test on its own, as part of kunit based on [0].
I also looked at the vDSO selftests and those all print the TAP headers on
their own. The same for all of the x86 selftests I looked at.

> The problem with adding TAP headers is two-fold:
> 
> 1. It becomes hard to read the ouptput if test is run using
>    the framework a. make -C timens run_tests b. cd timens; make run_tests

Isn't this what "KSFT_TAP_LEVEL" was added for in
commit 10f531f67482 ("selftests: kselftest framework: add handling for TAP header level") ?

> 2. These headers become bested and TAP doesn't like nested TAP headers.
>    My undestansing is parses will have problems.

They are also prefixed with a "#", so they should be ignored by parsers.

> I am adding Tim Bird to the discussion.
> 
> Tim, is this still an issue? Are nested TAP headers still a problem.
> 
> Even if they aren't there is the issue of multiple TAP headers for
> each test when we run the test using framework.

The suppression was actually there at some point and got removed by Kees in
commit f41c322f17ec ("selftests: Remove KSFT_TAP_LEVEL").

[0] https://lore.kernel.org/all/20250217-kunit-kselftests-v1-0-42b4524c3b0a@linutronix.de/