Message-ID: <7727e5d4f035c04d03ba274ad8b7fb8bc7da696c.camel@crowdstrike.com>
Date: Wed, 12 Feb 2025 22:31:21 +0000
From: Martin Kelly <martin.kelly@...wdstrike.com>
To: "andrii.nakryiko@...il.com" <andrii.nakryiko@...il.com>
CC: "mykolal@...com" <mykolal@...com>, "shuah@...nel.org" <shuah@...nel.org>,
        "eddyz87@...il.com" <eddyz87@...il.com>,
        "song@...nel.org" <song@...nel.org>,
        Mark Fontana <mark.fontana@...wdstrike.com>,
        "john.fastabend@...il.com"
	<john.fastabend@...il.com>,
        "yonghong.song@...ux.dev"
	<yonghong.song@...ux.dev>,
        "linux-kernel@...r.kernel.org"
	<linux-kernel@...r.kernel.org>,
        "andrii@...nel.org" <andrii@...nel.org>,
        "kpsingh@...nel.org" <kpsingh@...nel.org>,
        "martin.lau@...ux.dev"
	<martin.lau@...ux.dev>,
        "ast@...nel.org" <ast@...nel.org>,
        "bpf@...r.kernel.org" <bpf@...r.kernel.org>,
        "sdf@...ichev.me"
	<sdf@...ichev.me>,
        "daniel@...earbox.net" <daniel@...earbox.net>,
        "linux-kselftest@...r.kernel.org" <linux-kselftest@...r.kernel.org>,
        "Slava
 Imameev" <slava.imameev@...wdstrike.com>,
        "jolsa@...nel.org"
	<jolsa@...nel.org>,
        "haoluo@...gle.com" <haoluo@...gle.com>
Subject: Re: Re: Re: Re: [PATCH 2/2] libbpf: BPF programs dynamic loading and
 attaching

On Mon, 2025-02-10 at 16:06 -0800, Andrii Nakryiko wrote:
> > Tracking associated maps for a program is not necessary. As long as
> > the last BPF program using the BPF map is unloaded, the kernel will
> > automatically free not-anymore-referenced BPF map. Note that
> > bpf_object itself will keep FDs for BPF maps, so you'd need to make
> > sure to do bpf_object__close() to release those references.
> > 
> > But if you are going to ask to re-create BPF maps next time BPF
> > program is loaded... Well, I'll say you are asking for a bit too >
> > much,
> > tbh. If you want to be *that* sophisticated, it shouldn't be too
> > hard
> > for you to get all this information from BPF program's
> > instructions.
> > 

We really are that sophisticated (see below for more details). We could
scan program instructions, but we'd then tie our logic to BPF
implementation details and duplicate logic already present in libbpf
(https://elixir.bootlin.com/linux/v6.13.2/source/tools/lib/bpf/libbpf.c#L6087
). Obviously this *can* be done but it's not at all ideal from an
application perspective.


> > > > 
> > bpf_object is the unit of coherence in libbpf, so I don't see us
> > refcounting maps between bpf_objects. Kernel is doing refcounting
> > based on FDs, so see if you can use that.
> > 

I can understand that. That said, I think if there's no logic across
objects, and bpf_object access is not thread-safe, it puts us into a
tough situation:
- Complex refcounting, code scanning, etc to keep consistency when
manipulating maps used by multiple programs.
- Parallel loading not being well-balanced, if we split programs across
objects.

We could alternatively write our own custom loader, but then we’d have
to duplicate much of the useful logic that libbpf already implements:
skeleton generation, map/program association, embedding programs into
ELFs, loading logic and kernel probing, etc. We’d like some way to
handle dynamic/parallel loading without having to replicate all the
advantages libbpf grants us.

> > 
> > 
> > Is 100 just a nicely looking rather large number, or do you really
> > have 100 different BPF programs? Why so many and are they really
> > all
> > unique?
> > 
> > Asking because if it's just a way to attach BPF program doing more
> > or
> > less uniform set of actions for different hooks, then perhaps there
> > are better ways to do this without having to duplicating BPF
> > programs
> > so much (like BPF cookie, multi-kprobes, etc, etc)

100 is not an arbitrary number; we have that and higher (~200 is a good
current estimate, and that grows as new product features are added).
The programs are really doing different things. We also have to support
a wide range of kernels, handling cases like: "on this kernel range,
trampolines aren't supported, so use kretprobes with a context map for
function args instead of fexit, but on newer kernels just use an fexit
hook."

The use case here is that our security monitoring agent leverages eBPF
as its foundational technology to gather telemetry from the kernel. As
part of that, we hook many different kernel subsystems (process,
memory, filesystem, network, etc), tying them together and tracking
with maps. So we legitimately have a very large number of programs all
doing different work. For products of this scale, it increases security
and performance to load this set of programs and their maps in an
optimized, parallel fashion and subsequently change the loaded set of
programs and maps dynamically without disturbing the rest of the
application.

Powered by blists - more mailing lists