linux-kernel - Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAEf4BzYvZsjSpsDHXAuZ9G3=r4e27+c_LDpSUampw-fTfKA2=g@mail.gmail.com>
Date: Thu, 15 Jan 2026 09:58:31 -0800
From: Andrii Nakryiko <andrii.nakryiko@...il.com>
To: Alexis Lothoré (eBPF Foundation) <alexis.lothore@...tlin.com>
Cc: Andrii Nakryiko <andrii@...nel.org>, Eduard Zingerman <eddyz87@...il.com>, 
	Alexei Starovoitov <ast@...nel.org>, Daniel Borkmann <daniel@...earbox.net>, 
	Martin KaFai Lau <martin.lau@...ux.dev>, Song Liu <song@...nel.org>, 
	Yonghong Song <yonghong.song@...ux.dev>, John Fastabend <john.fastabend@...il.com>, 
	KP Singh <kpsingh@...nel.org>, Stanislav Fomichev <sdf@...ichev.me>, Hao Luo <haoluo@...gle.com>, 
	Jiri Olsa <jolsa@...nel.org>, Shuah Khan <shuah@...nel.org>, ebpf@...uxfoundation.org, 
	Bastien Curutchet <bastien.curutchet@...tlin.com>, 
	Thomas Petazzoni <thomas.petazzoni@...tlin.com>, linux-kernel@...r.kernel.org, 
	bpf@...r.kernel.org, linux-kselftest@...r.kernel.org
Subject: Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
<alexis.lothore@...tlin.com> wrote:
>
> Hello,
> this series is part of the larger effort aiming to convert all
> standalone tests to the CI runners so that they are properly executed on
> patches submission.
>
> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
> and so they do not integrate well in test_progs. This series proposes to

Can you elaborate why they do not integrate well? In my mind,
test_progs should be the only runner into which we invest effort
(parallel tests, all the different filtering, etc; why would we have
to reimplement subsets of this). The fact that we have test_maps and
test_verifier is historical and if we had enough time we'd merge all
of them into test_progs.

What exactly in test_progs would prevent us from implementing bpftool
test runner?

> introduce a new runner to support those tests. This new runner is
> heavily inspired from test_progs, with slightly less features.
>
> - First commit move the assert macros exposed by test_progs into a
>   shared header so that they can be used by the new runner
> - Second commit introduces the new runner, as well as a first test
> - Third commit adds a second test
> - Fourth commit drops the legacy scripts corresponding to those new
>   tests
>
> The series only adds the runner, but it is not enough to have it running
> in CI. The github actions need to be updated both in [1] and [2], and I
> am not sure how it si supposed to be contributed (the part needed in [1]
> must likely be integrated in the commit that is systematically brought
> on top of the bpf-next_base branch). Still, an example of test_bpftool
> run can be found in [3], in which I have forked and updated the needed
> Github actions ([4])
>
> [1] https://github.com/kernel-patches/bpf.git
> [2] https://github.com/libbpf/ci
> [3] https://github.com/kernel-patches/bpf/pull/10711
> [4] https://github.com/Tropicao/libbpf-ci
>
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@...tlin.com>
> ---
> Alexis Lothoré (eBPF Foundation) (4):
>       bpf/selftests: move assert macros into a dedicated header
>       bpf/selftests: introduce bptool test runner and a first test
>       selftests/bpf: add bpftool map manipulations tests
>       selftests/bpf: remove converted bpftool test scripts
>
>  tools/testing/selftests/bpf/.gitignore             |   1 +
>  tools/testing/selftests/bpf/Makefile               |  16 +-
>  tools/testing/selftests/bpf/assert_helpers.h       | 231 ++++++++++++
>  tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++
>  tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +
>  .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
>  .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++
>  .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++
>  tools/testing/selftests/bpf/test_bpftool.c         | 126 +++++++
>  tools/testing/selftests/bpf/test_bpftool.h         |  36 ++
>  tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
>  .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
>  tools/testing/selftests/bpf/test_progs.h           | 226 +-----------
>  13 files changed, 1041 insertions(+), 711 deletions(-)
> ---
> base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
> change-id: 20251212-bpftool-tests-bb165c4cceb8
>
> Best regards,
> --
> Alexis Lothoré (eBPF Foundation) <alexis.lothore@...tlin.com>
>