netdev - Re: [PATCH v2 bpf-next 00/11] libbpf: split BTF support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 5 Nov 2020 10:52:54 +0100
From:   Jesper Dangaard Brouer <brouer@...hat.com>
To:     Andrii Nakryiko <andrii@...nel.org>
Cc:     brouer@...hat.com, <bpf@...r.kernel.org>, <netdev@...r.kernel.org>,
        <ast@...com>, <daniel@...earbox.net>, <kernel-team@...com>,
        Saeed Mahameed <saeed@...nel.org>,
        David Ahern <dsahern@...nel.org>,
        Anton Protopopov <aspsk2@...il.com>
Subject: Re: [PATCH v2 bpf-next 00/11] libbpf: split BTF support

On Wed, 4 Nov 2020 20:33:50 -0800
Andrii Nakryiko <andrii@...nel.org> wrote:

> This patch set adds support for generating and deduplicating split BTF. This
> is an enhancement to the BTF, which allows to designate one BTF as the "base
> BTF" (e.g., vmlinux BTF), and one or more other BTFs as "split BTF" (e.g.,
> kernel module BTF), which are building upon and extending base BTF with extra
> types and strings.
> 
> Once loaded, split BTF appears as a single unified BTF superset of base BTF,
> with continuous and transparent numbering scheme. This allows all the existing
> users of BTF to work correctly and stay agnostic to the base/split BTFs
> composition.  The only difference is in how to instantiate split BTF: it
> requires base BTF to be alread instantiated and passed to btf__new_xxx_split()
> or btf__parse_xxx_split() "constructors" explicitly.
> 
> This split approach is necessary if we are to have a reasonably-sized kernel
> module BTFs. By deduping each kernel module's BTF individually, resulting
> module BTFs contain copies of a lot of kernel types that are already present
> in vmlinux BTF. Even those single copies result in a big BTF size bloat. On my
> kernel configuration with 700 modules built, non-split BTF approach results in
> 115MBs of BTFs across all modules. With split BTF deduplication approach,
> total size is down to 5.2MBs total, which is on part with vmlinux BTF (at
> around 4MBs). This seems reasonable and practical. As to why we'd need kernel
> module BTFs, that should be pretty obvious to anyone using BPF at this point,
> as it allows all the BTF-powered features to be used with kernel modules:
> tp_btf, fentry/fexit/fmod_ret, lsm, bpf_iter, etc.

I love to see this work going forward.

My/Our (+Saeed +Ahern) use-case is for NIC-driver kernel modules.  I
want drivers to define a BTF struct that describe a meta-data area that
can be consumed/used by XDP, also available during xdp_frame to SKB
transition, which happens in net-core. So, I hope BTF-IDs are also
"available" from core kernel code?

 
> This patch set is a pre-requisite to adding split BTF support to pahole, which
> is a prerequisite to integrating split BTF into the Linux kernel build setup
> to generate BTF for kernel modules. The latter will come as a follow-up patch
> series once this series makes it to the libbpf and pahole makes use of it.
> 
> Patch #4 introduces necessary basic support for split BTF into libbpf APIs.
> Patch #8 implements minimal changes to BTF dedup algorithm to allow
> deduplicating split BTFs. Patch #11 adds extra -B flag to bpftool to allow to
> specify the path to base BTF for cases when one wants to dump or inspect split
> BTF. All the rest are refactorings, clean ups, bug fixes and selftests.
> 
> v1->v2:
>   - addressed Song's feedback.
-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer