netdev - Re: [PATCH v4 bpf-next 0/7] Fix, clean up, and revamp selftests/bpf Makefile

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 17 Oct 2019 12:18:51 -0700
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     Jakub Sitnicki <jakub@...udflare.com>
Cc:     Andrii Nakryiko <andrii.nakryiko@...il.com>,
        Andrii Nakryiko <andriin@...com>, bpf <bpf@...r.kernel.org>,
        Network Development <netdev@...r.kernel.org>,
        Alexei Starovoitov <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Kernel Team <kernel-team@...com>
Subject: Re: [PATCH v4 bpf-next 0/7] Fix, clean up, and revamp selftests/bpf Makefile

On Thu, Oct 17, 2019 at 1:09 AM Jakub Sitnicki <jakub@...udflare.com> wrote:
>
> On Thu, Oct 17, 2019 at 08:52 AM CEST, Andrii Nakryiko wrote:
> > On Wed, Oct 16, 2019 at 9:28 PM Alexei Starovoitov
> > <alexei.starovoitov@...il.com> wrote:
> >>
> >> On Wed, Oct 16, 2019 at 4:49 AM Andrii Nakryiko <andriin@...com> wrote:
> >> >
> >> > This patch set extensively revamps selftests/bpf's Makefile to generalize test
> >> > runner concept and apply it uniformly to test_maps and test_progs test
> >> > runners, along with test_progs' few build "flavors", exercising various ways
> >> > to build BPF programs.
> >> >
> >> > As we do that, we fix dependencies between various phases of test runners, and
> >> > simplify some one-off rules and dependencies currently present in Makefile.
> >> > test_progs' flavors are now built into root $(OUTPUT) directory and can be run
> >> > without any extra steps right from there. E.g., test_progs-alu32 is built and
> >> > is supposed to be run from $(OUTPUT). It will cd into alu32/ subdirectory to
> >> > load correct set of BPF object files (which are different from the ones built
> >> > for test_progs).
> >> >
> >> > Outline:
> >> > - patch #1 teaches test_progs about flavor sub-directories;
> >> > - patch #2 fixes one of CO-RE tests to not depend strictly on process name;
> >> > - patch #3 changes test_maps's usage of map_tests/tests.h to be the same as
> >> >   test_progs' one;
> >> > - patch #4 adds convenient short `make test_progs`-like targets to build only
> >> >   individual tests, if necessary;
> >> > - patch #5 is a main patch in the series; it uses a bunch of make magic
> >> >   (mainly $(call) and $(eval)) to define test runner "skeleton" and apply it
> >> >   to 4 different test runners, lots more details in corresponding commit
> >> >   description;
> >> > - patch #6 does a bit of post-clean up for test_queue_map and test_stack_map
> >> >   BPF programs;
> >> > - patch #7 cleans up test_libbpf.sh/test_libbpf_open superseded by test_progs.
> >> >
> >> > v3->v4:
> >> > - remove accidentally checked in binaries;
> >>
> >> something really odd here.
> >> Before the patchset ./test_progs -n 27 passes
> >> after the patch it simply hangs.
> >> Though strace -f ./test_progs -n 27 passes.
> >> Any idea?
> >
> > Interesting. For me test_progs -n27 passes by itself, whether with or
> > without Makefile changes. But when run together with #8
> > flow_dissector_reattach, it fails with
> > "(prog_tests/sockopt_inherit.c:28: errno: Network is unreachable) Fail
> > to connect to server", even without Makefile changes. It doesn't hang,
> > but the test has server and client threads being coordinated, so I
> > wouldn't be surprised that under some specific timing and error
> > conditions it can hang.
> >
> > I bisected this failure to f97eea1756f3 ("selftests/bpf: Check that
> > flow dissector can be re-attached"), that's when
> > flow_dissector_reattach test was added. So apparently there is some
> > bad interaction there.
> >
> > So I suspect my Makefile changes have nothing to do with this, it
> > would be really bizarre...
> >
> > Jakub, do you mind checking as well?
>
> This is my fault. flow_dissector_reattach test is not returning to the
> netns we started in. Sorry about the oversight. Let me post a fix.

Thanks for the fix.
Now it all works nicely and selftests/bpf build time improvement
is dramatic.

Applied the set to bpf-next.
Thanks