netdev - Re: [PATCH bpf-next] selftests/bpf: add general instructions for test execution

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LRH.2.21.2005192224560.31696@localhost>
Date:   Tue, 19 May 2020 22:34:25 +0100 (BST)
From:   Alan Maguire <alan.maguire@...cle.com>
To:     Alexei Starovoitov <alexei.starovoitov@...il.com>
cc:     Alan Maguire <alan.maguire@...cle.com>, shuah@...nel.org,
        ast@...nel.org, daniel@...earbox.net, andriin@...com,
        bpf@...r.kernel.org, kafai@...com, songliubraving@...com,
        yhs@...com, john.fastabend@...il.com, kpsingh@...omium.org,
        linux-kselftest@...r.kernel.org, netdev@...r.kernel.org
Subject: Re: [PATCH bpf-next] selftests/bpf: add general instructions for
 test execution

On Tue, 19 May 2020, Alexei Starovoitov wrote:

> On Mon, May 18, 2020 at 12:23:10PM +0100, Alan Maguire wrote:
> > Getting a clean BPF selftests run involves ensuring latest trunk LLVM/clang
> > are used, pahole is recent (>=1.16) and config matches the specified
> > config file as closely as possible.  Document all of this in the general
> > README.rst file.  Also note how to work around timeout failures.
> > 
> > Signed-off-by: Alan Maguire <alan.maguire@...cle.com>
> > ---
> >  tools/testing/selftests/bpf/README.rst | 46 ++++++++++++++++++++++++++++++++++
> >  1 file changed, 46 insertions(+)
> > 
> > diff --git a/tools/testing/selftests/bpf/README.rst b/tools/testing/selftests/bpf/README.rst
> > index 0f67f1b..b00eebb 100644
> > --- a/tools/testing/selftests/bpf/README.rst
> > +++ b/tools/testing/selftests/bpf/README.rst
> > @@ -1,6 +1,52 @@
> >  ==================
> >  BPF Selftest Notes
> >  ==================
> > +First verify the built kernel config options match the config options
> > +specified in the config file in this directory.  Test failures for
> > +unknown helpers, inability to find BTF etc will be observed otherwise.
> > +
> > +To ensure the maximum number of tests pass, it is best to use the latest
> > +trunk LLVM/clang, i.e.
> > +
> > +git clone https://github.com/llvm/llvm-project
> > +
> > +Build/install trunk LLVM:
> > +
> > +.. code-block:: bash
> > +  git clone https://github.com/llvm/llvm-project
> > +  cd llvm-project
> > +  mkdir build/llvm
> > +  cd build/llvm
> > +  cmake ../../llvm/
> > +  make
> > +  sudo make install
> > +  cd ../../
> > +
> > +Build/install trunk clang:
> > +
> > +.. code-block:: bash
> > +  mkdir -p build/clang
> > +  cd build/clang
> > +  cmake ../../clang
> > +  make
> > +  sudo make install
> > +
> 
> these instructions are obsolete and partially incorrect.
> May be refer to Documentation/bpf/bpf_devel_QA.rst instead?
>

Sure; looks like there are up-to-date sections there on
running BPF selftests and building LLVM manually.  Perhaps
I should add the notes about pahole etc there too?
I should also have noted that without an up-to-date iproute2
failures will be observed also.
  
> > +When building the kernel with CONFIG_DEBUG_INFO_BTF, pahole
> > +version 16 or later is also required for BTF function
> > +support. pahole can be built from the source at
> > +
> > +https://github.com/acmel/dwarves
> > +
> > +It is often available in "dwarves/libdwarves" packages also,
> > +but be aware that versions prior to 1.16 will fail with
> > +errors that functions cannot be found in BTF.
> > +
> > +When running selftests, the default timeout of 45 seconds
> > +can be exceeded by some tests.  We can override the default
> > +timeout via a "settings" file; for example:
> > +
> > +.. code-block:: bash
> > +  echo "timeout=120" > tools/testing/selftests/bpf/settings
> 
> Is it really the case?
> I've never seen anything like this.
> 

When running via "make run_tests" on baremetal systems I
see test timeouts pretty consistently; e.g. from a bpf tree test
run yesterday:

not ok 6 selftests: bpf: test_progs # TIMEOUT
not ok 31 selftests: bpf: test_tunnel.sh # TIMEOUT
not ok 38 selftests: bpf: test_lwt_ip_encap.sh # TIMEOUT
not ok 40 selftests: bpf: test_tc_tunnel.sh # TIMEOUT
not ok 42 selftests: bpf: test_xdping.sh # TIMEOUT
not ok 43 selftests: bpf: test_bpftool_build.sh # TIMEOUT

These will only occur if running via "make run_tests",
so running tests individually would not trigger these
failures.

Alan