netdev - Re: [PATCH bpf-next 1/3] bpftool: add extra CO-RE mode to btf dump command

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAEf4Bzayg2UZi1H1NZaFgAUabtS5a=-yCE7NsUmtaO7kS5CJmw@mail.gmail.com>
Date:   Fri, 20 Dec 2019 09:40:31 -0800
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     Alexei Starovoitov <alexei.starovoitov@...il.com>
Cc:     Andrii Nakryiko <andriin@...com>, bpf <bpf@...r.kernel.org>,
        Networking <netdev@...r.kernel.org>,
        Alexei Starovoitov <ast@...com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Kernel Team <kernel-team@...com>
Subject: Re: [PATCH bpf-next 1/3] bpftool: add extra CO-RE mode to btf dump command

On Thu, Dec 19, 2019 at 2:04 PM Alexei Starovoitov
<alexei.starovoitov@...il.com> wrote:
>
> On Thu, Dec 19, 2019 at 01:07:38PM -0800, Andrii Nakryiko wrote:
> > On Thu, Dec 19, 2019 at 9:06 AM Alexei Starovoitov
> > <alexei.starovoitov@...il.com> wrote:
> > >
> > > On Wed, Dec 18, 2019 at 11:06:56PM -0800, Andrii Nakryiko wrote:
> > > > +     if (core_mode) {
> > > > +             printf("#if defined(__has_attribute) && __has_attribute(preserve_access_index)\n");
> > > > +             printf("#define __CLANG_BPF_CORE_SUPPORTED\n");
> > > > +             printf("#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)\n");
> > > > +             printf("#endif\n\n");
> > >
> > > I think it's dangerous to automatically opt-out when clang is not new enough.
> > > bpf prog will compile fine, but it will be missing co-re relocations.
> > > How about doing something like:
> > >   printf("#ifdef NEEDS_CO_RE\n");
> > >   printf("#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)\n");
> > >   printf("#endif\n\n");
> > > and emit it always when 'format c'.
> > > Then on the program side it will look:
> > > #define NEEDS_CO_RE
> > > #include "vmlinux.h"
> > > If clang is too old there will be a compile time error which is a good thing.
> > > Future features will have different NEEDS_ macros.
> >
> > Wouldn't it be cleaner to separate vanilla C types dump vs
> > CO-RE-specific one? I'd prefer to have them separate and not require
> > every application to specify this #define NEEDS_CO_RE macro.
> > Furthermore, later we probably are going to add some additional
> > auto-generated types, definitions, etc, so plain C types dump and
> > CO-RE-specific one will deviate quite a bit. So it feels cleaner to
> > separate them now instead of polluting `format c` with irrelevant
> > noise.
>
> Say we do this 'format core' today then tomorrow another tweak to vmlinux.h
> would need 'format core2' ? I think adding new format to bpftool for every

No, not at all, it will stay within `format core`. If we need to do
some parameterized tweak to BPF CO-RE-targeted vmlinux.h, then we'll
have to add this parameter (even though I'd try to avoid
parameterizing it as much as possible, of course).


> little feature will be annoying to users. I think the output should stay as
> 'format c' and that format should be extensible/customizable by bpf progs via
> #define NEEDS_FEATURE_X. Then these features can grow without a need to keep
> adding new cmd line args. This preserve_access_index feature makes up for less
> than 1% difference in generated vmlinux.h. If some feature extension would
> drastically change generated .h then it would justify new 'format'. This one is
> just a small tweak. Also #define NEEDS_CO_RE is probably too broad. I think

This one is a small line-number-wise. But the big difference between
`format c` and `format core` is that the latter assumes we are dumping
*vmlinux's BTF* for use with *BPF CO-RE from BPF side*. `format c`
doesn't make any assumptions and faithfully dumps whatever BTF
information is provided, which can be some other BPF program, or just
any userspace program, on which pahole -J was executed.

This assumption is why I think we should separate those two formats.
For `format core` we can start auto-generating extra helper types,
similarly how BCC auto-generates them for tracepoint, for example.

Technically, sure, we can always guard everything behind extra
#ifdefs, but think about dumping BTF type info for your small BPF
program, and instead of seeing clean dump of types, you see all those
crazy #ifdefs and weird #pragma clang's, extra attributes and so on.
Not a great user experience for sure.


> #define CLANG_NEEDS_TO_EMIT_RELO would be more precise and less ambiguous.