netdev - Re: [PATCH bpf-next 5/6] selftests/bpf: replace test_progs and test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAEf4BzbG9u-hhgX1HDf1PavfWv1fc-hs_DD_6khemerxi=qxew@mail.gmail.com>
Date:   Tue, 15 Oct 2019 20:34:24 -0700
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     Alexei Starovoitov <ast@...com>
Cc:     Andrii Nakryiko <andriin@...com>,
        "bpf@...r.kernel.org" <bpf@...r.kernel.org>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "daniel@...earbox.net" <daniel@...earbox.net>,
        Kernel Team <Kernel-team@...com>
Subject: Re: [PATCH bpf-next 5/6] selftests/bpf: replace test_progs and
 test_maps w/ general rule

On Tue, Oct 15, 2019 at 4:50 PM Andrii Nakryiko
<andrii.nakryiko@...il.com> wrote:
>
> On Tue, Oct 15, 2019 at 4:41 PM Alexei Starovoitov <ast@...com> wrote:
> >
> > On 10/15/19 3:03 PM, Andrii Nakryiko wrote:
> > > Define test runner generation meta-rule that codifies dependencies
> > > between test runner, its tests, and its dependent BPF programs. Use that
> > > for defining test_progs and test_maps test-runners. Also additionally define
> > > 3 flavors of test_progs:
> > > - alu32, which builds BPF programs with 32-bit registers codegen;
> > > - bpf_gcc, which build BPF programs using GCC, if it supports BPF target;
> > > - native, which uses a mix of native Clang target and BPF target for LLC.
> >
> > Great improvement, but it's taking it too far.
> > (clang  -I. -I/data/users/ast/net-next/tools/testing/selftests/bpf -g
> > -D__TARGET_ARCH_x86 -I. -I./include/uapi
> > -I/data/users/ast/net-next/tools/include/uapi
> > -I/data/users/ast/net-next/tools/lib/bpf
> > -I/data/users/ast/net-next/tools/testing/selftests/usr/include
> > -idirafter /usr/local/include -idirafter
> > /data/users/ast/llvm/bld/lib/clang/10.0.0/include -idirafter
> > /usr/include -Wno-compare-distinct-pointer-types -O2 -emit-llvm -c
> > progs/test_core_reloc_existence.c -o - || echo "BPF obj compilation
> > failed") | llc -march=bpf -mcpu=probe   -filetype=obj -o
> > /data/users/ast/net-next/tools/testing/selftests/bpf/native/test_core_reloc_existence.o
> > progs/test_core_reloc_existence.c:47:18: error: use of unknown builtin
> > '__builtin_preserve_field_info' [-Wimplicit-function-declaration]
> >          out->a_exists = bpf_core_field_exists(in->a);
>
> Do you use latest clang that supports __builtin_preserve_field_info()?
> All the flavors are building just fine for me with latest clang.

Ok, so I didn't have __builtin_preserve_field_info() tests together
with these Makefile changes and thus test_progs-native was compiling
just fine for me. __builtin_preserve_access_index() is not
BPF-target-only built-in, so it was compiling fine for non-BPF target.

I've dropped test_progs-native and added back test_xdp.o override, but
now **after** we define generic rule for test_progs, which will cause
it to override previous test_xdp.o recipe. This causes make to emit
warning about rule re-definition, which I'm not excited about, but
avoiding it would require some further checks and filterings in
DEFINE_TEST_RUNNER_RULES just for the sake of avoiding this warning,
which seems to be an overkill...

I'm wondering if we can just drop that mixed native Clang/bpf LLC mode
altogether? But for now we have it with a warning in v2.

>
> >
> > native clang + llc is useful for old school tracing only (before CO-RE).
>
> Don't disagree (I actually have little context why we needed this
> special case at all), but I had no errors or warnings whatsoever. I
> think in this particular case it's not specific to test_progs-native
> build, can you please double-check on your side?