netdev - Re: [PATCHv3 net-next 3/3] tools: ynl: add YNL test framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aRWqKA5nUAySkJFX@fedora>
Date: Thu, 13 Nov 2025 09:51:36 +0000
From: Hangbin Liu <liuhangbin@...il.com>
To: Matthieu Baerts <matttbe@...nel.org>
Cc: Donald Hunter <donald.hunter@...il.com>, netdev@...r.kernel.org,
	Jakub Kicinski <kuba@...nel.org>,
	"David S. Miller" <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>, Paolo Abeni <pabeni@...hat.com>,
	Simon Horman <horms@...nel.org>, Jan Stancek <jstancek@...hat.com>,
	Asbjørn Sloth Tønnesen <ast@...erby.net>,
	Stanislav Fomichev <sdf@...ichev.me>,
	Ido Schimmel <idosch@...dia.com>,
	Guillaume Nault <gnault@...hat.com>,
	Sabrina Dubroca <sd@...asysnail.net>,
	Petr Machata <petrm@...dia.com>
Subject: Re: [PATCHv3 net-next 3/3] tools: ynl: add YNL test framework

On Thu, Nov 13, 2025 at 10:21:41AM +0100, Matthieu Baerts wrote:
> >>> +	if [ $$failed -eq 0 ]; then \
> >>> +		echo "All tests passed!"; \
> >>> +	else \
> >>> +		echo "$$failed test(s) failed!"; \
> >>
> >> AFAICS this will never be reported since the scripts only ever exit 0.
> >> The message is also a bit misleading since it would be the count of
> >> scripts that failed, not individual tests.
> >>
> >> It would be great if the scripts exited with the number of test failures
> >> so the make file could report a total.
> > 
> > Oh, BTW, do you think if we should exit with the failed test number or just
> 
> I know these new tests are not in the selftests, but maybe "safer" to
> keep the same exit code to avoid being misinterpreted?
> 
>   KSFT_PASS=0
>   KSFT_FAIL=1
>   KSFT_XFAIL=2
>   KSFT_XPASS=3
>   KSFT_SKIP=4

Yes, that's why I ask about the return code. I also prefer use the same exit
code with selftest.

> 
> If there is a need to know which tests have failed, why not using (K)TAP
> format for the output?

I feel it's too heavy to copy the (K)TAP format here. I would just using the
exit code unless Jakub ask to using the specific output format

Thanks
Hangbin