netdev - Re: [PATCH v3 bpf-next 0/3] libbpf: BTF writer APIs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Mon, 28 Sep 2020 19:26:29 -0700
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     Andrii Nakryiko <andriin@...com>
Cc:     bpf <bpf@...r.kernel.org>,
        Network Development <netdev@...r.kernel.org>,
        Alexei Starovoitov <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Andrii Nakryiko <andrii.nakryiko@...il.com>,
        Kernel Team <kernel-team@...com>,
        Arnaldo Carvalho de Melo <acme@...hat.com>
Subject: Re: [PATCH v3 bpf-next 0/3] libbpf: BTF writer APIs

On Mon, Sep 28, 2020 at 7:06 PM Andrii Nakryiko <andriin@...com> wrote:
>
> This patch set introduces a new set of BTF APIs to libbpf that allow to
> conveniently produce BTF types and strings. These APIs will allow libbpf to do
> more intrusive modifications of program's BTF (by rewriting it, at least as of
> right now), which is necessary for the upcoming libbpf static linking. But
> they are complete and generic, so can be adopted by anyone who has a need to
> produce BTF type information.
>
> One such example outside of libbpf is pahole, which was actually converted to
> these APIs (locally, pending landing of these changes in libbpf) completely
> and shows reduction in amount of custom pahole code necessary and brings nice
> savings in memory usage (about 370MB reduction at peak for my kernel
> configuration) and even BTF deduplication times (one second reduction,
> 23.7s -> 22.7s). Memory savings are due to avoiding pahole's own copy of
> "uncompressed" raw BTF data. Time reduction comes from faster string
> search and deduplication by relying on hashmap instead of BST used by pahole's
> own code. Consequently, these APIs are already tested on real-world
> complicated kernel BTF, but there is also pretty extensive selftest doing
> extra validations.
>
> Selftests in patch #3 add a set of generic ASSERT_{EQ,STREQ,ERR,OK} macros
> that are useful for writing shorter and less repretitive selftests. I decided
> to keep them local to that selftest for now, but if they prove to be useful in
> more contexts we should move them to test_progs.h. And few more (e.g.,
> inequality tests) macros are probably necessary to have a more complete set.
>
> Cc: Arnaldo Carvalho de Melo <acme@...hat.com>
>
> v2->v3:
>   - resending original patches #7-9 as patches #1-3 due to merge conflict;

Applied. Thanks