netdev - Re: [PATCH v4 bpf-next 5/7] selftests/bpf: replace test_progs and test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAEf4BzZvNQwcn3=sUHjnVfGzAMkfECpiJ7=YEDWSnLFZD7xeCA@mail.gmail.com>
Date:   Thu, 17 Oct 2019 10:50:54 -0700
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     Andrii Nakryiko <andriin@...com>
Cc:     bpf <bpf@...r.kernel.org>, Networking <netdev@...r.kernel.org>,
        Alexei Starovoitov <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Kernel Team <kernel-team@...com>
Subject: Re: [PATCH v4 bpf-next 5/7] selftests/bpf: replace test_progs and
 test_maps w/ general rule

On Tue, Oct 15, 2019 at 11:01 PM Andrii Nakryiko <andriin@...com> wrote:
>
> Define test runner generation meta-rule that codifies dependencies
> between test runner, its tests, and its dependent BPF programs. Use that
> for defining test_progs and test_maps test-runners. Also additionally define
> 2 flavors of test_progs:
> - alu32, which builds BPF programs with 32-bit registers codegen;
> - bpf_gcc, which build BPF programs using GCC, if it supports BPF target.
>
> Overall, this is accomplished through $(eval)'ing a set of generic
> rules, which defines Makefile targets dynamically at runtime. See
> comments explaining the need for 2 $(evals), though.
>
> For each test runner we have (test_maps and test_progs, currently), and,
> optionally, their flavors, the logic of build process is modeled as
> follows (using test_progs as an example):
> - all BPF objects are in progs/:
>   - BPF object's .o file is built into output directory from
>     corresponding progs/.c file;
>   - all BPF objects in progs/*.c depend on all progs/*.h headers;
>   - all BPF objects depend on bpf_*.h helpers from libbpf (but not
>     libbpf archive). There is an extra rule to trigger bpf_helper_defs.h
>     (re-)build, if it's not present/outdated);
>   - build recipe for BPF object can be re-defined per test runner/flavor;
> - test files are built from prog_tests/*.c:
>   - all such test file objects are built on individual file basis;
>   - currently, every single test file depends on all BPF object files;
>     this might be improved in follow up patches to do 1-to-1 dependency,
>     but allowing to customize this per each individual test;
>   - each test runner definition can specify a list of extra .c and .h
>     files to be built along test files and test runner binary; all such
>     headers are becoming automatic dependency of each test .c file;
>   - due to test files sometimes embedding (using .incbin assembly
>     directive) contents of some BPF objects at compilation time, which are
>     expected to be in CWD of compiler, compilation for test file object does
>     cd into test runner's output directory; to support this mode all the
>     include paths are turned into absolute paths using $(abspath) make
>     function;
> - prog_tests/test.h is automatically (re-)generated with an entry for
>   each .c file in prog_tests/;
> - final test runner binary is linked together from test object files and
>   extra object files, linking together libbpf's archive as well;
> - it's possible to specify extra "resource" files/targets, which will be
>   copied into test runner output directory, if it differes from
>   Makefile-wide $(OUTPUT). This is used to ensure btf_dump test cases and
>   urandom_read binary is put into a test runner's CWD for tests to find
>   them in runtime.
>
> For flavored test runners, their output directory is a subdirectory of
> common Makefile-wide $(OUTPUT) directory with flavor name used as
> subdirectory name.
>
> BPF objects targets might be reused between different test runners, so
> extra checks are employed to not double-define them. Similarly, we have
> redefinition guards for output directories and test headers.
>
> test_verifier follows slightly different patterns and is simple enough
> to not justify generalizing TEST_RUNNER_DEFINE/TEST_RUNNER_DEFINE_RULES
> further to accomodate these differences. Instead, rules for
> test_verifier are minimized and simplified, while preserving correctness
> of dependencies.
>
> Signed-off-by: Andrii Nakryiko <andriin@...com>
> ---

BTW, if correctness and DRY-ness argument is not strong enough, these
changes makes clean rebuild from scratch about 2x faster for me:

BEFORE: `make clean && time make -j50` is 14-15 seconds
AFTER: `make clean && time make -j50` is 7-8 seconds


[...]