linux-kernel - Re: [PATCH] selftests/bpf: Use kselftest skip code for skipped tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87h7fdg8pr.fsf@cloudflare.com>
Date:   Wed, 25 Aug 2021 11:44:00 +0200
From:   Jakub Sitnicki <jakub@...udflare.com>
To:     Po-Hsu Lin <po-hsu.lin@...onical.com>
Cc:     linux-kernel@...r.kernel.org, bpf@...r.kernel.org,
        netdev@...r.kernel.org, linux-kselftest@...r.kernel.org,
        hawk@...nel.org, kuba@...nel.org, davem@...emloft.net,
        kpsingh@...nel.org, john.fastabend@...il.com, yhs@...com,
        songliubraving@...com, kafai@...com, andrii@...nel.org,
        daniel@...earbox.net, ast@...nel.org, skhan@...uxfoundation.org
Subject: Re: [PATCH] selftests/bpf: Use kselftest skip code for skipped tests

On Mon, Aug 23, 2021 at 05:01 AM CEST, Po-Hsu Lin wrote:
> There are several test cases in the bpf directory are still using
> exit 0 when they need to be skipped. Use kselftest framework skip
> code instead so it can help us to distinguish the return status.
>
> Criterion to filter out what should be fixed in bpf directory:
>   grep -r "exit 0" -B1 | grep -i skip
>
> This change might cause some false-positives if people are running
> these test scripts directly and only checking their return codes,
> which will change from 0 to 4. However I think the impact should be
> small as most of our scripts here are already using this skip code.
> And there will be no such issue if running them with the kselftest
> framework.
>
> Signed-off-by: Po-Hsu Lin <po-hsu.lin@...onical.com>
> ---
>  tools/testing/selftests/bpf/test_bpftool_build.sh | 5 ++++-
>  tools/testing/selftests/bpf/test_xdp_meta.sh      | 5 ++++-
>  tools/testing/selftests/bpf/test_xdp_vlan.sh      | 7 +++++--
>  3 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/test_bpftool_build.sh b/tools/testing/selftests/bpf/test_bpftool_build.sh
> index ac349a5..b6fab1e 100755
> --- a/tools/testing/selftests/bpf/test_bpftool_build.sh
> +++ b/tools/testing/selftests/bpf/test_bpftool_build.sh
> @@ -1,6 +1,9 @@
>  #!/bin/bash
>  # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
>  
> +# Kselftest framework requirement - SKIP code is 4.
> +ksft_skip=4
> +
>  case $1 in
>  	-h|--help)
>  		echo -e "$0 [-j <n>]"
> @@ -22,7 +25,7 @@ KDIR_ROOT_DIR=$(realpath $PWD/$SCRIPT_REL_DIR/../../../../)
>  cd $KDIR_ROOT_DIR
>  if [ ! -e tools/bpf/bpftool/Makefile ]; then
>  	echo -e "skip:    bpftool files not found!\n"
> -	exit 0
> +	exit $ksft_skip
>  fi
>  
>  ERROR=0

This bit has been fixed a couple days ago by a similar change:

https://lore.kernel.org/bpf/20210820025549.28325-1-lizhijian@cn.fujitsu.com

> diff --git a/tools/testing/selftests/bpf/test_xdp_meta.sh b/tools/testing/selftests/bpf/test_xdp_meta.sh
> index 637fcf4..fd3f218 100755
> --- a/tools/testing/selftests/bpf/test_xdp_meta.sh
> +++ b/tools/testing/selftests/bpf/test_xdp_meta.sh
> @@ -1,5 +1,8 @@
>  #!/bin/sh
>  
> +# Kselftest framework requirement - SKIP code is 4.
> +ksft_skip=4
> +
>  cleanup()
>  {
>  	if [ "$?" = "0" ]; then

Would consider making it read-only:

  readonly KSFT_SKIP=4

[...]