netdev - Re: [PATCH net-next 0/4] selftests: kselftest

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Wed, 14 Feb 2024 11:09:50 +0100
From: Jakub Sitnicki <jakub@...udflare.com>
To: Jakub Kicinski <kuba@...nel.org>
Cc: shuah@...nel.org, keescook@...omium.org,
 linux-kselftest@...r.kernel.org, netdev@...r.kernel.org
Subject: Re: [PATCH net-next 0/4] selftests: kselftest_harness: support
 using xfail

On Tue, Feb 13, 2024 at 07:44 AM -08, Jakub Kicinski wrote:
> Hi!
>
> When running selftests for our subsystem in our CI we'd like all
> tests to pass. Currently some tests use SKIP for cases they
> expect to fail, because the kselftest_harness limits the return
> codes to pass/fail/skip.
>
> Clean up and support the use of the full range of ksft exit codes
> under kselftest_harness.
>
> To avoid conflicts and get the functionality into the networking
> tree ASAP I'd like to put the patches on shared branch so that
> both linux-kselftest and net-next can pull it in. Shuah, please
> LMK if that'd work for you, and if so which -rc should I base
> the branch on. Or is merging directly into net-next okay?
>
> Jakub Kicinski (4):
>   selftests: kselftest_harness: pass step via shared memory
>   selftests: kselftest_harness: use KSFT_* exit codes
>   selftests: kselftest_harness: support using xfail
>   selftests: ip_local_port_range: use XFAIL instead of SKIP
>
>  tools/testing/selftests/kselftest_harness.h   | 67 ++++++++++++++-----
>  .../selftests/net/ip_local_port_range.c       |  2 +-
>  2 files changed, 52 insertions(+), 17 deletions(-)

Nice!
We've been ignoring skipped tests in our internal CI.
But this is the wrong approach, as you point out.

For the series:

Tested-by: Jakub Sitnicki <jakub@...udflare.com>