netdev - Re: [PATCH v2 bpf-next] selftests/bpf: add whitelist/blacklist of test names to test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAADnVQ++qp83cW_1M4WyD8qiGzyPBDC5a-DJLLwgU97rh_EvSg@mail.gmail.com>
Date:   Wed, 15 Jan 2020 18:42:22 -0800
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     Andrii Nakryiko <andriin@...com>
Cc:     bpf <bpf@...r.kernel.org>,
        Network Development <netdev@...r.kernel.org>,
        Alexei Starovoitov <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Andrii Nakryiko <andrii.nakryiko@...il.com>,
        Kernel Team <kernel-team@...com>, Julia Kartseva <hex@...com>
Subject: Re: [PATCH v2 bpf-next] selftests/bpf: add whitelist/blacklist of
 test names to test_progs

On Wed, Jan 15, 2020 at 6:16 PM Andrii Nakryiko <andriin@...com> wrote:
>
> Add ability to specify a list of test name substrings for selecting which
> tests to run. So now -t is accepting a comma-separated list of strings,
> similarly to how -n accepts a comma-separated list of test numbers.
>
> Additionally, add ability to blacklist tests by name. Blacklist takes
> precedence over whitelist. Blacklisting is important for cases where it's
> known that some tests can't pass (e.g., due to perf hardware events that are
> not available within VM). This is going to be used for libbpf testing in
> Travis CI in its Github repo.
>
> Example runs with just whitelist and whitelist + blacklist:
>
>   $ sudo ./test_progs -tattach,core/existence
>   #1 attach_probe:OK
>   #6 cgroup_attach_autodetach:OK
>   #7 cgroup_attach_multi:OK
>   #8 cgroup_attach_override:OK
>   #9 core_extern:OK
>   #10/44 existence:OK
>   #10/45 existence___minimal:OK
>   #10/46 existence__err_int_sz:OK
>   #10/47 existence__err_int_type:OK
>   #10/48 existence__err_int_kind:OK
>   #10/49 existence__err_arr_kind:OK
>   #10/50 existence__err_arr_value_type:OK
>   #10/51 existence__err_struct_type:OK
>   #10 core_reloc:OK
>   #19 flow_dissector_reattach:OK
>   #60 tp_attach_query:OK
>   Summary: 8/8 PASSED, 0 SKIPPED, 0 FAILED
>
>   $ sudo ./test_progs -tattach,core/existence -bcgroup,flow/arr
>   #1 attach_probe:OK
>   #9 core_extern:OK
>   #10/44 existence:OK
>   #10/45 existence___minimal:OK
>   #10/46 existence__err_int_sz:OK
>   #10/47 existence__err_int_type:OK
>   #10/48 existence__err_int_kind:OK
>   #10/51 existence__err_struct_type:OK
>   #10 core_reloc:OK
>   #60 tp_attach_query:OK
>   Summary: 4/6 PASSED, 0 SKIPPED, 0 FAILED
>
> Cc: Julia Kartseva <hex@...com>
> Signed-off-by: Andrii Nakryiko <andriin@...com>

Nice. Even things like "test_progs -n 11-16 -b exit" work as expected.
Applied. Thanks!