netdev - Re: [PATCH btf 0/3] Add BTF types deduplication algorithm

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Mon, 4 Feb 2019 17:34:13 -0800
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     Daniel Borkmann <daniel@...earbox.net>
Cc:     Andrii Nakryiko <andriin@...com>, netdev@...r.kernel.org,
        Alexei Starovoitov <ast@...com>, Yonghong Song <yhs@...com>,
        Martin Lau <kafai@...com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        kernel-team@...com
Subject: Re: [PATCH btf 0/3] Add BTF types deduplication algorithm

On Mon, Feb 4, 2019 at 5:32 PM Daniel Borkmann <daniel@...earbox.net> wrote:
>
> Hi Andrii,
>
> On 01/31/2019 07:58 AM, Andrii Nakryiko wrote:
> > This patch series adds BTF deduplication algorithm to libbpf. This algorithm
> > allows to take BTF type information containing duplicate per-compilation unit
> > information and reduce it to equivalent set of BTF types with no duplication without
> > loss of information. It also deduplicates strings and removes those strings that
> > are not referenced from any BTF type (and line information in .BTF.ext section,
> > if any).
> >
> > Algorithm also resolves struct/union forward declarations into concrete BTF types
> > across multiple compilation units to facilitate better deduplication ratio. If
> > undesired, this resolution can be disabled through specifying corresponding options.
> >
> > When applied to BTF data emitted by pahole's DWARF->BTF converter, it reduces
> > the overall size of .BTF section by about 65x, from about 112MB to 1.75MB, leaving
> > only 29247 out of initial 3073497 BTF type descriptors.
> >
> > Algorithm with minor differences and preliminary results before FUNC/FUNC_PROTO
> > support is also described more verbosely at:
> > https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html
> >
> > Andrii Nakryiko (3):
> >   btf: extract BTF type size calculation
> >   btf: add BTF types deduplication algorithm
> >   selftests/btf: add initial BTF dedup tests
> >
> >  tools/lib/bpf/btf.c                    | 1851 +++++++++++++++++++++++-
> >  tools/lib/bpf/btf.h                    |   11 +
> >  tools/lib/bpf/libbpf.map               |    3 +
> >  tools/testing/selftests/bpf/test_btf.c |  535 ++++++-
> >  4 files changed, 2333 insertions(+), 67 deletions(-)
>
> This would need a proper rebase for bpf-next so that it applies w/o
> bigger conflicts. Please also change the libbpf.map and place the newly
> exported functions under LIBBPF_0.0.2 (as everything under 0.0.1 was from
> prior released kernel).

Done. Thanks!

>
> Thanks,
> Daniel