linux-kernel - Re: [PATCH] selftests/bpf: Use kselftest skip code for skipped tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAMy_GT8=vViQOGANARcTb4qyB4635nLC=XAFZTijchou+Aa9Dg@mail.gmail.com>
Date:   Wed, 29 Sep 2021 13:22:54 +0800
From:   Po-Hsu Lin <po-hsu.lin@...onical.com>
To:     Jakub Sitnicki <jakub@...udflare.com>
Cc:     linux-kernel <linux-kernel@...r.kernel.org>, bpf@...r.kernel.org,
        netdev@...r.kernel.org, linux-kselftest@...r.kernel.org,
        hawk@...nel.org, Jakub Kicinski <kuba@...nel.org>,
        David Miller <davem@...emloft.net>, kpsingh@...nel.org,
        john.fastabend@...il.com, yhs@...com, songliubraving@...com,
        kafai@...com, andrii@...nel.org, daniel@...earbox.net,
        ast@...nel.org, Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH] selftests/bpf: Use kselftest skip code for skipped tests

On Wed, Aug 25, 2021 at 5:44 PM Jakub Sitnicki <jakub@...udflare.com> wrote:
>
> On Mon, Aug 23, 2021 at 05:01 AM CEST, Po-Hsu Lin wrote:
> > There are several test cases in the bpf directory are still using
> > exit 0 when they need to be skipped. Use kselftest framework skip
> > code instead so it can help us to distinguish the return status.
> >
> > Criterion to filter out what should be fixed in bpf directory:
> >   grep -r "exit 0" -B1 | grep -i skip
> >
> > This change might cause some false-positives if people are running
> > these test scripts directly and only checking their return codes,
> > which will change from 0 to 4. However I think the impact should be
> > small as most of our scripts here are already using this skip code.
> > And there will be no such issue if running them with the kselftest
> > framework.
> >
> > Signed-off-by: Po-Hsu Lin <po-hsu.lin@...onical.com>
> > ---
> >  tools/testing/selftests/bpf/test_bpftool_build.sh | 5 ++++-
> >  tools/testing/selftests/bpf/test_xdp_meta.sh      | 5 ++++-
> >  tools/testing/selftests/bpf/test_xdp_vlan.sh      | 7 +++++--
> >  3 files changed, 13 insertions(+), 4 deletions(-)
> >
> > diff --git a/tools/testing/selftests/bpf/test_bpftool_build.sh b/tools/testing/selftests/bpf/test_bpftool_build.sh
> > index ac349a5..b6fab1e 100755
> > --- a/tools/testing/selftests/bpf/test_bpftool_build.sh
> > +++ b/tools/testing/selftests/bpf/test_bpftool_build.sh
> > @@ -1,6 +1,9 @@
> >  #!/bin/bash
> >  # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> >
> > +# Kselftest framework requirement - SKIP code is 4.
> > +ksft_skip=4
> > +
> >  case $1 in
> >       -h|--help)
> >               echo -e "$0 [-j <n>]"
> > @@ -22,7 +25,7 @@ KDIR_ROOT_DIR=$(realpath $PWD/$SCRIPT_REL_DIR/../../../../)
> >  cd $KDIR_ROOT_DIR
> >  if [ ! -e tools/bpf/bpftool/Makefile ]; then
> >       echo -e "skip:    bpftool files not found!\n"
> > -     exit 0
> > +     exit $ksft_skip
> >  fi
> >
> >  ERROR=0
>
> This bit has been fixed a couple days ago by a similar change:
>
> https://lore.kernel.org/bpf/20210820025549.28325-1-lizhijian@cn.fujitsu.com
>
Hello Jakub,

Thanks for the feedback, I have submit a v2 patch for this:
https://lore.kernel.org/bpf/20210929051250.13831-1-po-hsu.lin@canonical.com/

Cheers

> > diff --git a/tools/testing/selftests/bpf/test_xdp_meta.sh b/tools/testing/selftests/bpf/test_xdp_meta.sh
> > index 637fcf4..fd3f218 100755
> > --- a/tools/testing/selftests/bpf/test_xdp_meta.sh
> > +++ b/tools/testing/selftests/bpf/test_xdp_meta.sh
> > @@ -1,5 +1,8 @@
> >  #!/bin/sh
> >
> > +# Kselftest framework requirement - SKIP code is 4.
> > +ksft_skip=4
> > +
> >  cleanup()
> >  {
> >       if [ "$?" = "0" ]; then
>
> Would consider making it read-only:
>
>   readonly KSFT_SKIP=4
>
> [...]