Date:   Thu, 17 Dec 2020 17:46:42 +0000 (GMT)
From:   Alan Maguire <alan.maguire@...cle.com>
To:     Alexei Starovoitov <alexei.starovoitov@...il.com>
cc:     Andrii Nakryiko <andrii.nakryiko@...il.com>,
        Alan Maguire <alan.maguire@...cle.com>,
        Alexei Starovoitov <ast@...com>,
        Andrii Nakryiko <andrii@...nel.org>, bpf <bpf@...r.kernel.org>,
        Networking <netdev@...r.kernel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Kernel Team <kernel-team@...com>
Subject: Re: one prog multi fentry. Was: [PATCH bpf-next] libbpf: support
 module BTF for BPF_TYPE_ID_TARGET CO-RE relocation



On Wed, 16 Dec 2020, Alexei Starovoitov wrote:

> > > $ ksnoop "ip_send_skb(skb->sk)"
> > >
> > > ...will trace the skb->sk value.  The user-space side of the program
> > > matches the function/arg name and looks up the referenced type, setting it
> > > in the function's map.  For field references such as skb->sk, it also
> > > records offset and whether that offset is a pointer (as is the case for
> > > skb->sk) - in such cases we need to read the offset value via bpf_probe_read()
> > > and use it in bpf_snprintf_btf() along with the referenced type.  Only a
> > > single simple reference like the above is supported currently, but
> > > multiple levels of reference could be made to work too.
> 
> Alan,
> 
> I'm not sure why the last example is so different form the first two.
> I think ksnoop tool will generate the program on the fly, right?

Nope, the BPF program is hard-coded; it adapts to different functions
through use of the map entries describing function signatures and their
BTF ids, and other associated tracing info.  The aim is to provide a
generic tracing tool which displays kernel function arguments but
doesn't require LLVM/clang on the target, just a kernel built with BTF 
and libbpf.  Sorry this wasn't clearer in my explanation; I'm working
on rewriting the code and will send it out ASAP.

> So it can generate normal LDX insn with CO-RE relocation (instead of bpf_probe_read)
> to access skb->sk. It can also add relo for that LDX to point to
> struct sk_buff's btf_id defined inside prog's BTF.
> The 'sk' offset inside bpf program and inside BTF can be anything: 0, 4, ...
> libbpf relocation logic will find the right offset in kernel's sk_buff.
> If ksnoop doesn't have an ability to parse vmlinux.h file or kernel's BTF
> it can 'cheat'.
> If the cmdline looks like:
> $ ksnoop "ip_send_skb(skb->sk)"
> It can generate BTF:
> struct sk_buff {
>    struct sock *sk;
> };
> 
> If cmdline looks like:
> $ ksnoop "ip_send_skb(skb->sock)"
> It can generate BTF:
> struct sk_buff {
>    struct sock *sock;
> };
> Obviously there is no 'sock' field inside kernel's struct sk_buff, but tool
> doesn't need to care. It can let libbpf do the checking and match
> fields properly.
> 
> > > into that a bit more if you don't mind because I think some form of
> > > user-space-specified BTF ids may be the easiest approach for more flexible
> > > generic tracing that covers more than function arguments.
> 
> I think you're trying to figure out kernel's btf_ids in ksnoop tool.

Yep.

> I suggest to leave that job to libbpf. Generate local BTFs in ksnoop
> with CO-RE relocs and let libbpf handle insn patching.
> No FDs to worry about from ksnoop side either.
> 

The current approach doesn't rely on instruction patching outside
of limited CORE use around struct pt_regs fields (args, IP, etc)
which shouldn't require LLVM/clang availability on the target system. 
I'll try and get it ready for RFC submission by the weekend so you
can see more details of the approach.

Thanks!

Alan

Powered by blists - more mailing lists