netdev - Re: [PATCH v2 bpf-next 00/11] libbpf: split BTF support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <105c48d56a550af6e0008b4b5867eb51764d41c9.camel@nvidia.com>
Date:   Thu, 5 Nov 2020 19:38:17 +0000
From:   Saeed Mahameed <saeedm@...dia.com>
To:     "andrii.nakryiko@...il.com" <andrii.nakryiko@...il.com>,
        "brouer@...hat.com" <brouer@...hat.com>
CC:     "aspsk2@...il.com" <aspsk2@...il.com>, "ast@...com" <ast@...com>,
        "dsahern@...nel.org" <dsahern@...nel.org>,
        "daniel@...earbox.net" <daniel@...earbox.net>,
        "kernel-team@...com" <kernel-team@...com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "bpf@...r.kernel.org" <bpf@...r.kernel.org>,
        "andrii@...nel.org" <andrii@...nel.org>
Subject: Re: [PATCH v2 bpf-next 00/11] libbpf: split BTF support

On Thu, 2020-11-05 at 11:16 -0800, Andrii Nakryiko wrote:
> > > This split approach is necessary if we are to have a reasonably-
> > > sized kernel
> > > module BTFs. By deduping each kernel module's BTF individually,
> > > resulting
> > > module BTFs contain copies of a lot of kernel types that are
> > > already present
> > > in vmlinux BTF. Even those single copies result in a big BTF size
> > > bloat. On my
> > > kernel configuration with 700 modules built, non-split BTF
> > > approach results in
> > > 115MBs of BTFs across all modules. With split BTF deduplication
> > > approach,
> > > total size is down to 5.2MBs total, which is on part with vmlinux
> > > BTF (at
> > > around 4MBs). This seems reasonable and practical. As to why we'd
> > > need kernel
> > > module BTFs, that should be pretty obvious to anyone using BPF at
> > > this point,
> > > as it allows all the BTF-powered features to be used with kernel
> > > modules:
> > > tp_btf, fentry/fexit/fmod_ret, lsm, bpf_iter, etc.
> > I love to see this work going forward.
> 
> 
> Thanks.
> 
> 
> 
> > My/Our (+Saeed +Ahern) use-case is for NIC-driver kernel modules. 
> > I
> > want drivers to define a BTF struct that describe a meta-data area
> > that
> > can be consumed/used by XDP, also available during xdp_frame to SKB
> > transition, which happens in net-core. So, I hope BTF-IDs are also
> > "available" from core kernel code?
> 
> 
> I'll probably need a more specific example to understand what exactly
> 
> you are asking and how you see everything working together, sorry.
> 
> 

BTF-IDs can be made available for kernel/drivers, I've wrote a small
patch for this a while ago.

https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=6c1cb83629226889d6fadd3ba694e827fca3e247

So the basic use case is that :
1- driver kernel/registers a BTF format (one or more).
2- Userland queries driver's registered BTF to be able understand the
kernel/driver buffers format.

driver example of using this infrastructure:
https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=9c24657d6cb3a7852c2e948dc9782f3f39b60104

User Queries driver's XDP metadata BTF format:
https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=6a117e2d9196f58de7cf067741e84ec242af27f6

Dump it as C header style 
https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=
8bd99626879bff28379707ac3a2c3bb94fd5b410

And then use it in your XDP program to parse packets meta data passed
from this specific driver. ( i mean no real parsing is required, you
just point to the meta data buffer with the metadata btf formatted C
strucuter).


> 
> If you are asking about support for using BTF_ID_LIST() macro in a
> 
> kernel module, then right now we don't call resolve_btfids on
> modules,
> 
> so it's not supported there yet. It's trivial to add, but we'll
> 
> probably need to teach resolve_btfids to understand split BTF. We can
> 
> do that separately after the basic "infra" lands, though.