netdev - Re: Kernel 5.5.4 build fail for BPF-selftests with latest LLVM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Wed, 19 Feb 2020 14:26:36 -0800
From:   Andrii Nakryiko <andrii.nakryiko@...il.com>
To:     Daniel Díaz <daniel.diaz@...aro.org>
Cc:     Jesper Dangaard Brouer <brouer@...hat.com>,
        Shuah Khan <shuah@...nel.org>,
        Alexei Starovoitov <alexei.starovoitov@...il.com>,
        Andrii Nakryiko <andriin@...com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        BPF-dev-list <bpf@...r.kernel.org>,
        Daniel Borkmann <borkmann@...earbox.net>,
        David Miller <davem@...emloft.net>,
        LKML <linux-kernel@...r.kernel.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Anders Roxell <anders.roxell@...aro.org>,
        Toke Høiland-Jørgensen <toke@...hat.com>
Subject: Re: Kernel 5.5.4 build fail for BPF-selftests with latest LLVM

On Wed, Feb 19, 2020 at 1:59 PM Daniel Díaz <daniel.diaz@...aro.org> wrote:
>
> Hello!
>
> On Wed, 19 Feb 2020 at 14:06, Jesper Dangaard Brouer <brouer@...hat.com> wrote:
> >
> > On Wed, 19 Feb 2020 10:38:45 -0800
> > Andrii Nakryiko <andrii.nakryiko@...il.com> wrote:
> >
> > > On Wed, Feb 19, 2020 at 10:29 AM Jesper Dangaard Brouer
> > > <brouer@...hat.com> wrote:
> > > >
> > > > On Wed, 19 Feb 2020 09:38:50 -0800
> > > > Andrii Nakryiko <andrii.nakryiko@...il.com> wrote:
> > > >
> > > > > On Wed, Feb 19, 2020 at 9:04 AM Jesper Dangaard Brouer
> > > > > <brouer@...hat.com> wrote:
> > > > > >
> > > > > > On Wed, 19 Feb 2020 08:41:27 -0800
> > > > > > Alexei Starovoitov <alexei.starovoitov@...il.com> wrote:
> > > > > >
> > > > > > > On Wed, Feb 19, 2020 at 4:30 AM Jesper Dangaard Brouer
> > > > > > > <brouer@...hat.com> wrote:
> > > > > > > >
> > > > > > > > I'm willing to help out, such that we can do either version or feature
> > > > > > > > detection, to either skip compiling specific test programs or at least
> > > > > > > > give users a proper warning of they are using a too "old" LLVM version.
> > > > > > > ...
> > > > > > > > progs/test_core_reloc_bitfields_probed.c:47:13: error: use of unknown builtin '__builtin_preserve_field_info' [-Wimplicit-function-declaration]
> > > > > > > >         out->ub1 = BPF_CORE_READ_BITFIELD_PROBED(in, ub1);
> > > > > > >
> > > > > > > imo this is proper warning message already.
> > > > > >
> > > > > > This is an error, not a warning.  The build breaks as the make process stops.
> > > > > >
> > > > >
> > > > > Latest Clang was a requirement for building and running all selftests
> > > > > for a long time now. There were few previous discussions on mailing
> > > > > list about this and each time the conclusion was the same: latest
> > > > > Clang is a requirement for BPF selftests.
> > > >
> > > > The latest Clang is 9.0.1, and it doesn't build with that.
> > >
> > > Latest as in "latest built from sources".
> >
> > When I download a specific kernel release, how can I know what LLVM
> > git-hash or version I need (to use BPF-selftests)?
> >
> > Do you think it is reasonable to require end-users to compile their own
> > bleeding edge version of LLVM, to use BPF-selftests?
> >
> > I do hope that some end-users of BPF-selftests will be CI-systems.
> > That also implies that CI-system maintainers need to constantly do
> > "latest built from sources" of LLVM git-tree to keep up.  Is that a
> > reasonable requirement when buying a CI-system in the cloud?
>
> We [1] are end users of kselftests and many other test suites [2]. We
> run all of our testing on every git-push on linux-stable-rc, mainline,
> and linux-next -- approximately 1 million tests per week. We have a
> dedicated engineering team looking after this CI infrastructure and
> test results, and as such, I can wholeheartedly echo Jesper's
> sentiment here: We would really like to help kernel maintainers and
> developers by automatically testing their code in real hardware, but
> the BPF kselftests are difficult to work with from a CI perspective.
> We have caught and reported [3] many [4] build [5] failures [6] in the
> past for libbpf/Perf, but building is just one of the pieces. We are
> unable to run the entire BPF kselftests because only a part of the
> code builds, so our testing is very limited there.
>
> We hope that this situation can be improved and that our and everyone
> else's automated testing can help you guys too. For this to work out,
> we need some help.

Is it hard to make sure that your CIs install latest builds of Clang,
though? See [0], Clang has even latest Clang 11 snapshots available
(though BPF selftests need only Clang 10 right now). In fact, libbpf's
Github repo just got a support for building latest kernel + building
latest selftests + running selftests in QEMU, performed for each PR
([1]), in Travis CI. Making selftests silently being not built/run if
Clang is too old will just hide problems without anyone noticing.

  [0] http://apt.llvm.org/
  [1] https://travis-ci.org/libbpf/libbpf/jobs/651838387?utm_medium=notification&utm_source=github_status

>
> [1] https://lkft.linaro.org/
> [2] https://www.youtube.com/watch?v=R3H9fPhPf54&t=1m26s
> [3] https://lore.kernel.org/bpf/CA+G9fYtAQGwf=OoEvHwbJpitcfhpfhy-ar+6FRrWC_-ti7sUTg@mail.gmail.com/
> [4] https://lore.kernel.org/stable/CA+G9fYtxRoK6D1_oMf9zQj8MW0JtPdphDDO1NHcYQcoFNL5pjw@mail.gmail.com/
> [5] https://lore.kernel.org/bpf/CA+G9fYssgDcBkiNGSV7BmjE4Tj1j1_fa4VTJFv3N=2FHzewQLg@mail.gmail.com/
> [6] https://lore.kernel.org/bpf/CA+G9fYsK8zn3jqF=Wz6=8BBx4i1JTkv2h-LCbjE11UJkcz_NEA@mail.gmail.com/