Message-ID: <FA608BAA-B75D-4F56-B8E8-529468C89034@fb.com>
Date:   Fri, 30 Oct 2020 06:45:11 +0000
From:   Song Liu <songliubraving@...com>
To:     Andrii Nakryiko <andrii.nakryiko@...il.com>
CC:     Andrii Nakryiko <andrii@...nel.org>, bpf <bpf@...r.kernel.org>,
        Networking <netdev@...r.kernel.org>,
        Alexei Starovoitov <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Kernel Team <Kernel-team@...com>
Subject: Re: [PATCH bpf-next 00/11] libbpf: split BTF support



> On Oct 29, 2020, at 7:33 PM, Andrii Nakryiko <andrii.nakryiko@...il.com> wrote:
> 
> On Thu, Oct 29, 2020 at 5:33 PM Song Liu <songliubraving@...com> wrote:
>> 
>> 
>> 
>>> On Oct 28, 2020, at 5:58 PM, Andrii Nakryiko <andrii@...nel.org> wrote:
>>> 
>>> This patch set adds support for generating and deduplicating split BTF. This
>>> is an enhancement to the BTF, which allows to designate one BTF as the "base
>>> BTF" (e.g., vmlinux BTF), and one or more other BTFs as "split BTF" (e.g.,
>>> kernel module BTF), which are building upon and extending base BTF with extra
>>> types and strings.
>>> 
>>> Once loaded, split BTF appears as a single unified BTF superset of base BTF,
>>> with continuous and transparent numbering scheme. This allows all the existing
>>> users of BTF to work correctly and stay agnostic to the base/split BTFs
>>> composition.  The only difference is in how to instantiate split BTF: it
>>> requires base BTF to be alread instantiated and passed to btf__new_xxx_split()
>>> or btf__parse_xxx_split() "constructors" explicitly.
>>> 
>>> This split approach is necessary if we are to have a reasonably-sized kernel
>>> module BTFs. By deduping each kernel module's BTF individually, resulting
>>> module BTFs contain copies of a lot of kernel types that are already present
>>> in vmlinux BTF. Even those single copies result in a big BTF size bloat. On my
>>> kernel configuration with 700 modules built, non-split BTF approach results in
>>> 115MBs of BTFs across all modules. With split BTF deduplication approach,
>>> total size is down to 5.2MBs total, which is on part with vmlinux BTF (at
>>> around 4MBs). This seems reasonable and practical. As to why we'd need kernel
>>> module BTFs, that should be pretty obvious to anyone using BPF at this point,
>>> as it allows all the BTF-powered features to be used with kernel modules:
>>> tp_btf, fentry/fexit/fmod_ret, lsm, bpf_iter, etc.
>> 
>> Some high level questions. Do we plan to use split BTF for in-tree modules
>> (those built together with the kernel) or out-of-tree modules (those built
>> separately)? If it is for in-tree modules, is it possible to build split BTF
>> into vmlinux BTF?
> 
> It will be possible to use for both in-tree and out-of-tree. For
> in-tree, this will be integrated into the kernel build process. For
> out-of-tree, whoever builds their kernel module will need to invoke
> pahole -J with an extra flag pointing to the right vmlinux image (I
> haven't looked into the exact details of this integration, maybe there
> are already scripts in Linux repo that out-of-tree modules have to
> use, in such case we can add this integration there).

Thanks for the explanation. 

> 
> Merging all in-tree modules' BTFs into vmlinux's BTF defeats the
> purpose of the split BTF and will just increase the size of vmlinux
> BTF unnecessarily.

Is the purpose of split BTF to save memory used by module BTF? In the 
example above, I guess part of those 5.2MB will be loaded at run time, 
so the actual saving is less than 5.2MB. 5.2MB is really small for a 
decent system, e.g. ~0.03% of my laptop's main memory. 

Did I miss anything here? 

Song

Powered by blists - more mailing lists