netdev - Re: [PATCH net-next] selftests: net: report output format as TAP 13 in Python tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250302105059.5f35fc3b@kernel.org>
Date: Sun, 2 Mar 2025 10:50:59 -0800
From: Jakub Kicinski <kuba@...nel.org>
To: Matthieu Baerts <matttbe@...nel.org>
Cc: davem@...emloft.net, netdev@...r.kernel.org, edumazet@...gle.com,
 pabeni@...hat.com, andrew+netdev@...n.ch, horms@...nel.org,
 shuah@...nel.org, petrm@...dia.com, willemb@...gle.com,
 linux-kselftest@...r.kernel.org
Subject: Re: [PATCH net-next] selftests: net: report output format as TAP 13
 in Python tests

On Sat, 1 Mar 2025 11:52:06 +0100 Matthieu Baerts wrote:
> On 28/02/2025 19:00, Jakub Kicinski wrote:
> > The Python lib based tests report that they are producing
> > "KTAP version 1", but really we aren't making use of any
> > KTAP features, like subtests. Our output is plain TAP.
> > 
> > Report TAP 13 instead of KTAP 1, this is what mptcp tests do,  
> 
> Indeed, and also TC tests, and all the ones using kselftest_harness.h I
> think.
> 
> > and what NIPA knows how to parse best. For HW testing we need
> > precise subtest result tracking.  
> 
> I guess the best would be to have the kselftests infrastructure fully
> supporting KTAP, and then have a way for each test to print subtests
> correctly, not nested in the comments like it is done for the moment.
> But that looks harder to put in place, and the current solution is used
> in a few areas.

Right. So for additional context for folks less familiar with selftests
the two minor inconveniences which made "supporting KTAP" a less
exciting option are: (a) we run make run_tests which wraps things
in an outer TAP, so we'd end up with a hybrid, anyway; (b) NIPA
only supports one level of subtests.

Thanks for the review!