linux-kernel - Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAEf4BzbT-7iRezzNRQPvQpRDA3BmkesCSijT4mPXuWb2ua=9ag@mail.gmail.com>
Date: Fri, 16 Jan 2026 14:20:33 -0800
From: Andrii Nakryiko <andrii.nakryiko@...il.com>
To: Alexis Lothoré <alexis.lothore@...tlin.com>
Cc: Andrii Nakryiko <andrii@...nel.org>, Eduard Zingerman <eddyz87@...il.com>, 
	Alexei Starovoitov <ast@...nel.org>, Daniel Borkmann <daniel@...earbox.net>, 
	Martin KaFai Lau <martin.lau@...ux.dev>, Song Liu <song@...nel.org>, 
	Yonghong Song <yonghong.song@...ux.dev>, John Fastabend <john.fastabend@...il.com>, 
	KP Singh <kpsingh@...nel.org>, Stanislav Fomichev <sdf@...ichev.me>, Hao Luo <haoluo@...gle.com>, 
	Jiri Olsa <jolsa@...nel.org>, Shuah Khan <shuah@...nel.org>, ebpf@...uxfoundation.org, 
	Bastien Curutchet <bastien.curutchet@...tlin.com>, 
	Thomas Petazzoni <thomas.petazzoni@...tlin.com>, linux-kernel@...r.kernel.org, 
	bpf@...r.kernel.org, linux-kselftest@...r.kernel.org
Subject: Re: [PATCH bpf-next 0/4] selftests/bpf: add a new runner for bpftool tests

On Thu, Jan 15, 2026 at 11:57 PM Alexis Lothoré
<alexis.lothore@...tlin.com> wrote:
>
> Hi Andrii,
>
> On Thu Jan 15, 2026 at 6:58 PM CET, Andrii Nakryiko wrote:
> > On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
> > <alexis.lothore@...tlin.com> wrote:
> >>
> >> Hello,
> >> this series is part of the larger effort aiming to convert all
> >> standalone tests to the CI runners so that they are properly executed on
> >> patches submission.
> >>
> >> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
> >> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
> >> and so they do not integrate well in test_progs. This series proposes to
> >
> > Can you elaborate why they do not integrate well? In my mind,
> > test_progs should be the only runner into which we invest effort
> > (parallel tests, all the different filtering, etc; why would we have
> > to reimplement subsets of this). The fact that we have test_maps and
> > test_verifier is historical and if we had enough time we'd merge all
> > of them into test_progs.
> >
> > What exactly in test_progs would prevent us from implementing bpftool
> > test runner?
>
> I don't think there is any strong technical blocker preventing from
> integrating those tests directly into test_progs. That's rather about
> the fact that test_progs tests depends (almost) exclusively on
> libbpf/skeletons. Those bpftool tests rather need to directly execute

There are actually plenty of test in test_progs that do networking
setups, calling system() to launch various binaries, etc. So it never
was purely for libbpf/skeletons/whatnot.

So yeah, I think bpftool testing should still be implemented as one
(or many) test_progs tests (and maybe subtests), utilizing
test_progs's generic multi-process testing setup, filtering,
reporting, etc infrastructure. No need to add extra runners.

> bpftool and parse its stdout output, so I thought that it made sense to
> have a dedicated runner for this. If I'm wrong and so if those tests
> should rather be moved in the test_progs runner (eg to avoid duplicating
> the runner features), I'm fine with it. Any additional opinion on this
> is welcome.
>
> Thanks,
>
> Alexis
> --
> Alexis Lothoré, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>