netdev - Re: [PATCH bpf-next v2 6/9] bpf: iterators: install libbpf headers when building

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAEf4BzaG5Xs9Z3JC3e3fZORnkxByNd0_-MbjDWx1qNr9eBZGOw@mail.gmail.com>
Date:   Wed, 6 Oct 2021 10:44:54 -0700
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     Quentin Monnet <quentin@...valent.com>
Cc:     Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Andrii Nakryiko <andrii@...nel.org>,
        Networking <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>
Subject: Re: [PATCH bpf-next v2 6/9] bpf: iterators: install libbpf headers
 when building

On Tue, Oct 5, 2021 at 1:03 PM Quentin Monnet <quentin@...valent.com> wrote:
>
> On Mon, 4 Oct 2021 at 22:30, Quentin Monnet <quentin@...valent.com> wrote:
> >
> > On Mon, 4 Oct 2021 at 20:11, Andrii Nakryiko <andrii.nakryiko@...il.com> wrote:
> > >
> > > On Sat, Oct 2, 2021 at 3:12 PM Quentin Monnet <quentin@...valent.com> wrote:
> > > >
> > > > On Sat, 2 Oct 2021 at 21:27, Quentin Monnet <quentin@...valent.com> wrote:
> > > > >
> > > > > On Sat, 2 Oct 2021 at 00:20, Andrii Nakryiko <andrii.nakryiko@...il.com> wrote:
> > > > > >
> > > > > > On Fri, Oct 1, 2021 at 4:09 AM Quentin Monnet <quentin@...valent.com> wrote:
> > > > > > >
> > > > > > > API headers from libbpf should not be accessed directly from the
> > > > > > > library's source directory. Instead, they should be exported with "make
> > > > > > > install_headers". Let's make sure that bpf/preload/iterators/Makefile
> > > > > > > installs the headers properly when building.
> > > > >
> > > > > > >
> > > > > > > -$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)
> > > > > > > +$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile)            \
> > > > > > > +          | $(LIBBPF_OUTPUT) $(LIBBPF_INCLUDE)
> > > > > >
> > > > > > Would it make sense for libbpf's Makefile to create include and output
> > > > > > directories on its own? We wouldn't need to have these order-only
> > > > > > dependencies everywhere, right?
> > > > >
> > > > > Good point, I'll have a look at it.
> > > > > Quentin
> > > >
> > > > So libbpf already creates the include (and parent $(DESTDIR))
> > > > directory, so I can get rid of the related dependencies. But I don't
> > > > see an easy solution for the output directory for the object files.
> > > > The issue is that libbpf's Makefile includes
> > > > tools/scripts/Makefile.include, which checks $(OUTPUT) and errors out
> > >
> > > Did you check what benefits the use of tools/scripts/Makefile.include
> > > brings? Last time I had to deal with some non-trivial Makefile
> > > problem, this extra dance with tools/scripts/Makefile.include and some
> > > related complexities didn't seem very justified. So unless there are
> > > some very big benefits to having tool's Makefile.include included, I'd
> > > rather simplify libbpf's in-kernel Makefile and make it more
> > > straightforward. We have a completely independent separate Makefile
> > > for libbpf in Github, and I think it's more straightforward. Doesn't
> > > have to be done in this change, of course, but I was curious to hear
> > > your thoughts given you seem to have spent tons of time on this
> > > already.
> >
> > No, I haven't checked in details so far. I remember that several
> > elements defined in the Makefile.include are used in libbpf's
> > Makefile, and I stopped at that, because I thought that a refactoring
> > of the latter would be beyond the current set. But yes, I can have a
> > look at it and see if it's worth changing in a follow-up.
>
> Looking more at tools/scripts/Makefile.include: It's 160-line long and
> does not include any other Makefile, so there's nothing in it that we
> couldn't re-implement in libbpf's Makefile if necessary. This being
> said, it has a number of items that, I think, are good to keep there
> and share with the other tools. For example:
>
> - The $(EXTRA_WARNINGS) definitions
> - QUIET_GEN, QUIET_LINK, QUIET_CLEAN, which are not mandatory to have
> but integrate nicely with the way other tools (or kernel components)
> are built
> - Some overwrites for the toolchain, if $(LLVM) or $(CROSS_COMPILE) are set
>

I looked at Makefile again, I had bigger reservations about
tools/build/Makefile.include actually, as it causes some round-about
ways to do the actual build, e.g.:

$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS)
$(SHLIB_FLAGS)"

Like, what's going on here? What's $(build)? Everything can be
deciphered, but a simple operation of compiling one file at a time
becomes some maze of indirect make invocations... But that's a problem
for another day. So never mind.


> Thinking more about this, if we want to create the $(OUTPUT) directory
> in libbpf itself, we could maybe just enclose the check on its
> pre-existence in tools/scripts/Makefile.include with a dedicated
> variable ("ifneq ($(_skip_output_check),) ...") and set the latter in
> Makefile.include. This way we wouldn't have to change the current
> Makefile infra too much, and can keep the include.

_skip_output_check to bifurcate the behavior? I'd rather not.

>
> Quentin