linux-kernel - Re: [PATCH v4 bpf 2/2] bpf: selftests: add lwt redirect regression test cases

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAADnVQ+7akLU_rY7xe=LbvY=bLiN8C5Vk5m0iwbao6hVH9v88w@mail.gmail.com>
Date:   Mon, 31 Jul 2023 11:46:56 -0700
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     Jakub Sitnicki <jakub@...udflare.com>
Cc:     Martin KaFai Lau <martin.lau@...ux.dev>,
        Yan Zhai <yan@...udflare.com>, bpf <bpf@...r.kernel.org>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Andrii Nakryiko <andrii@...nel.org>,
        Song Liu <song@...nel.org>, Yonghong Song <yhs@...com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...nel.org>,
        Stanislav Fomichev <sdf@...gle.com>,
        Hao Luo <haoluo@...gle.com>, Jiri Olsa <jolsa@...nel.org>,
        "David S. Miller" <davem@...emloft.net>,
        Eric Dumazet <edumazet@...gle.com>,
        Jakub Kicinski <kuba@...nel.org>,
        Paolo Abeni <pabeni@...hat.com>,
        Mykola Lysenko <mykolal@...com>, Shuah Khan <shuah@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Network Development <netdev@...r.kernel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        kernel-team <kernel-team@...udflare.com>,
        Jordan Griege <jgriege@...udflare.com>,
        Markus Elfring <Markus.Elfring@....de>
Subject: Re: [PATCH v4 bpf 2/2] bpf: selftests: add lwt redirect regression
 test cases

On Mon, Jul 31, 2023 at 2:52 AM Jakub Sitnicki <jakub@...udflare.com> wrote:
>
> On Fri, Jul 28, 2023 at 03:47 PM -07, Martin KaFai Lau wrote:
> > On 7/25/23 6:09 PM, Yan Zhai wrote:
>
> [...]
>
> >> diff --git a/tools/testing/selftests/bpf/test_lwt_redirect.sh
> >> b/tools/testing/selftests/bpf/test_lwt_redirect.sh
> >> new file mode 100755
> >> index 000000000000..1b7b78b48174
> >> --- /dev/null
> >> +++ b/tools/testing/selftests/bpf/test_lwt_redirect.sh
> >
> > This has to be written in the test_progs infrastructure in C. Only test_progs is
> > run by the BPF CI. Take a look at other tests in prog_tests/. For example,
> > tc_redirect.c and xdp_metadata.c which are having setup in netns/link/...etc. It
> > currently has helpers to add tc qdisc and filter but not adding route yet which
> > could be a useful addition.
>
> Can we help make the BPF CI better so that it also runs other tests in
> addition test_progs?

Not really.
CI is not just running the test. It needs to understand the output,
pass it to UI, run in parallel, etc.
All the shell scripts are not suitable for long term CI exposure.

So I completely agree with Martin. No new shell scripts.
All selftests must be in test_progs.

> We have bpf selftests written in shell and even Python. These are
> sometimes the right tools for the job and make adding tests easier,
> IMHO. Network setup from C is verbose and tedious. Not to mention, hard
> to read through.

For comparison take a look at BPF CI code base and what it takes to run
the tests and process the output. There is plenty of work for CI ahead.