netdev - Re: [PATCH v3 bpf-next 0/9] Revamp test_progs as a test running framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Sat, 27 Jul 2019 22:41:22 -0700
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     Andrii Nakryiko <andriin@...com>
Cc:     bpf <bpf@...r.kernel.org>,
        Network Development <netdev@...r.kernel.org>,
        Alexei Starovoitov <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Stanislav Fomichev <sdf@...ichev.me>,
        Andrii Nakryiko <andrii.nakryiko@...il.com>,
        Kernel Team <kernel-team@...com>
Subject: Re: [PATCH v3 bpf-next 0/9] Revamp test_progs as a test running framework

On Sat, Jul 27, 2019 at 8:25 PM Andrii Nakryiko <andriin@...com> wrote:
>
> This patch set makes a number of changes to test_progs selftest, which is
> a collection of many other tests (and sometimes sub-tests as well), to provide
> better testing experience and allow to start convering many individual test
> programs under selftests/bpf into a single and convenient test runner.
>
> Patch #1 fixes issue with Makefile, which makes prog_tests/test.h compiled as
> a C code. This fix allows to change how test.h is generated, providing ability
> to have more control on what and how tests are run.
>
> Patch #2 changes how test.h is auto-generated, which allows to have test
> definitions, instead of just running test functions. This gives ability to do
> more complicated test run policies.
>
> Patch #3 adds `-t <test-name>` and `-n <test-num>` selectors to run only
> subset of tests.
>
> Patch #4 changes libbpf_set_print() to return previously set print callback,
> allowing to temporarily replace current print callback and then set it back.
> This is necessary for some tests that want more control over libbpf logging.
>
> Patch #5 sets up and takes over libbpf logging from individual tests to
> test_prog runner, adding -vv verbosity to capture debug output from libbpf.
> This is useful when debugging failing tests.
>
> Patch #6 furthers test output management and buffers it by default, emitting
> log output only if test fails. This give succinct and clean default test
> output. It's possible to bypass this behavior with -v flag, which will turn
> off test output buffering.
>
> Patch #7 adds support for sub-tests. It also enhances -t and -n selectors to
> both support ability to specify sub-test selectors, as well as enhancing
> number selector to accept sets of test, instead of just individual test
> number.
>
> Patch #8 converts bpf_verif_scale.c test to use sub-test APIs.
>
> Patch #9 converts send_signal.c tests to use sub-test APIs.
>
> v2->v3:
>   - fix buffered output rare unitialized value bug (Alexei);
>   - fix buffered output va_list reuse bug (Alexei);
>   - fix buffered output truncation due to interleaving zero terminators;

Looks great.
Applied. Thanks!