netdev - Re: [PATCH v3 bpf-next 5/7] libbpf: move bpf_{helpers,endian,tracing}.h into libbpf

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <62b1bc6b-8c8a-b766-6bfc-2fb16017d591@fb.com>
Date:   Fri, 4 Oct 2019 18:37:44 +0000
From:   Yonghong Song <yhs@...com>
To:     Jakub Kicinski <jakub.kicinski@...ronome.com>,
        Andrii Nakryiko <andrii.nakryiko@...il.com>
CC:     David Ahern <dsahern@...il.com>, Andrii Nakryiko <andriin@...com>,
        bpf <bpf@...r.kernel.org>, Networking <netdev@...r.kernel.org>,
        "Alexei Starovoitov" <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Kernel Team <Kernel-team@...com>, Jiri Benc <jbenc@...hat.com>
Subject: Re: [PATCH v3 bpf-next 5/7] libbpf: move
 bpf_{helpers,endian,tracing}.h into libbpf



On 10/4/19 11:30 AM, Jakub Kicinski wrote:
> On Fri, 4 Oct 2019 09:00:42 -0700, Andrii Nakryiko wrote:
>> On Fri, Oct 4, 2019 at 8:44 AM David Ahern <dsahern@...il.com> wrote:
>>>> I'm not following you; my interpretation of your comment seems like you
>>> are making huge assumptions.
>>>
>>> I build bpf programs for specific kernel versions using the devel
>>> packages for the specific kernel of interest.
>>
>> Sure, and you can keep doing that, just don't include bpf_helpers.h?
>>
>> What I was saying, though, especially having in mind tracing BPF
>> programs that need to inspect kernel structures, is that it's quite
>> impractical to have to build many different versions of BPF programs
>> for each supported kernel version and distribute them in binary form.
>> So people usually use BCC and do compilation on-the-fly using BCC's
>> embedded Clang.
>>
>> BPF CO-RE is providing an alternative, which will allow to pre-compile
>> your program once for many different kernels you might be running your
>> program on. There is tooling that eliminates the need for system
>> headers. Instead we pre-generate a single vmlinux.h header with all
>> the types/enums/etc, that are then used w/ BPF CO-RE to build portable
>> BPF programs capable of working on multiple kernel versions.
>>
>> So what I was pointing out there was that this vmlinux.h would be
>> ideally generated from latest kernel and not having latest
>> BPF_FUNC_xxx shouldn't be a problem. But see below about situation
>> being worse.
> 
> Surely for distroes tho - they would have kernel headers matching the
> kernel release they ship. If parts of libbpf from GH only work with
> the latest kernel, distroes should ship libbpf from the kernel source,
> rather than GH.
> 
>>>> Nevertheless, it is a problem and thanks for bringing it up! I'd say
>>>> for now we should still go ahead with this move and try to solve with
>>>> issue once bpf_helpers.h is in libbpf. If bpf_helpers.h doesn't work
>>>> for someone, it's no worse than it is today when users don't have
>>>> bpf_helpers.h at all.
>>>>   
>>>
>>> If this syncs to the github libbpf, it will be worse than today in the
>>> sense of compile failures if someone's header file ordering picks
>>> libbpf's bpf_helpers.h over whatever they are using today.
>>
>> Today bpf_helpers.h don't exist for users or am I missing something?
>> bpf_helpers.h right now are purely for selftests. But they are really
>> useful outside that context, so I'm making it available for everyone
>> by distributing with libbpf sources. If bpf_helpers.h doesn't work for
>> some specific use case, just don't use it (yet?).
>>
>> I'm still failing to see how it's worse than situation today.
> 
> Having a header which works today, but may not work tomorrow is going
> to be pretty bad user experience :( No matter how many warnings you put
> in the source people will get caught off guard by this :(
> 
> If you define the current state as "users can use all features of
> libbpf and nothing should break on libbpf update" (which is in my
> understanding a goal of the project, we bent over backwards trying
> to not break things) then adding this header will in fact make things
> worse. The statement in quotes would no longer be true, no?

distro can package bpf/btf uapi headers into libbpf package.
Users linking with libbpf.a/libbpf.so can use bpf/btf.h with include
path pointing to libbpf dev package include directory.
Could this work?