netdev - Re: bpftool cross-compilation for arm64

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CABWYdi09ZtYgMvptBO_G6gh_TC-joM4=cz5_KoTw7koHnMjhmg@mail.gmail.com>
Date:   Tue, 18 Dec 2018 16:40:02 -0800
From:   Ivan Babrou <ivan@...udflare.com>
To:     Stanislav Fomichev <sdf@...ichev.me>
Cc:     netdev@...r.kernel.org, jakub.kicinski@...ronome.com,
        Ignat Korchagin <ignat@...udflare.com>
Subject: Re: bpftool cross-compilation for arm64

On Tue, Dec 18, 2018 at 4:30 PM Stanislav Fomichev <sdf@...ichev.me> wrote:
>
> On 12/18, Ivan Babrou wrote:
> > Hey,
> >
> > Is it possible to cross-compile bpftool for arm64? I'm trying to do
> > the same thing we do for perf, but bpftool Makefile doesn't seem to
> > support CROSS_COMPILE variable, so one needs to pass
> > CC=aarch64-linux-gnu-gcc as well. Problems do not end there as it
> What error are you getting? Can you try setting the ARCH:
> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-

It just fails with the following:

$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-

Auto-detecting system features:
...                        libbfd: [ on  ]
...        disassembler-four-args: [ OFF ]

make[1]: Entering directory '/tmp/build/linux-4.19.10/tools/lib/bpf'
make[1]: Leaving directory '/tmp/build/linux-4.19.10/tools/lib/bpf'
  LINK     bpftool
/usr/bin/ld: /tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a(libbpf-in.o):
Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a(libbpf-in.o):
Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a(libbpf-in.o):
Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a(libbpf-in.o):
Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a(libbpf-in.o):
Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a(libbpf-in.o):
Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a(libbpf-in.o):
Relocations in generic ELF (EM: 183)
/tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a: error adding symbols:
File in wrong format
collect2: error: ld returned 1 exit status
Makefile:96: recipe for target 'bpftool' failed
make: *** [bpftool] Error 1

The reason is that CROSS_COMPILE doesn't do anything:

$ grep ^CC Makefile
CC = gcc

> [..]
> > requires EXTRA_LDFLAGS="-lz" too. With that I'm getting a long sad
> > error about missing stuff from libbfd:
> For this I've been waiting for the following patch to be merged
> first: https://lkml.org/lkml/2018/11/16/243
> Basically, on some platforms, -lbfd requires -liberty and/or -lz.
> We need to test each case individually for bpftool, but the feature
> tests are semi-broken without the patch above.

Adding -liberty doesn't help:

$ make ARCH=arm64 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
CC=aarch64-linux-gnu-gcc EXTRA_LDFLAGS="-lz -liberty"^C
(reverse-i-search)`QUI': make -C
/tmp/build/linux-4.19.10/tools/bpf/bpftool KBUILD_BUILD_TIMESTAMP="Mon
Sep 27 00:00:00 UTC 2010" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
CC=aarch64-linux-gnu-gcc EXTRA_LDFLAGS="-lz -liberty"
O=/tmp/build/arm64/bpftool
DESTDIR=/tmp/build/arm64/debian/bpftooltmp/usr install
LOCALVERSION=-cloudflare-2018.12.6-dev QUIET_LINK= ^CIET_CC=
ivan@...338:/tmp/build/linux-4.19.10/tools/bpf/bpftool$ make
ARCH=arm64 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
CC=aarch64-linux-gnu-gcc EXTRA_LDFLAGS="-lz -liberty" QUIET_LINK=
QUIET_CC=

Auto-detecting system features:
...                        libbfd: [ on  ]
...        disassembler-four-args: [ OFF ]

make[1]: Entering directory '/tmp/build/linux-4.19.10/tools/lib/bpf'
make[1]: Leaving directory '/tmp/build/linux-4.19.10/tools/lib/bpf'
aarch64-linux-gnu-gcc -O2 -W -Wall -Wextra -Wno-unused-parameter
-Wshadow -Wno-missing-field-initializers -DPACKAGE='"bpftool"'
-D__EXPORTED_HEADERS__ -I/tmp/build/linux-4.19.10/kernel/bpf/
-I/tmp/build/linux-4.19.10/tools/include
-I/tmp/build/linux-4.19.10/tools/include/uapi
-I/tmp/build/linux-4.19.10/tools/lib/bpf
-I/tmp/build/linux-4.19.10/tools/perf -DBPFTOOL_VERSION='"4.19.10"'
-DCOMPAT_NEED_REALLOCARRAY -lz -liberty -o bpftool map_perf_ring.o
xlated_dumper.o perf.o prog.o btf_dumper.o common.o cgroup.o main.o
json_writer.o cfg.o map.o jit_disasm.o disasm.o
/tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a -lelf -lbfd -lopcodes
/tmp/build/linux-4.19.10/tools/lib/bpf/libbpf.a
/lib/aarch64-linux-gnu/libbfd.a(elflink.o): In function
`elf_link_add_object_symbols':
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:4938:
undefined reference to `objalloc_free_block'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:5185:
undefined reference to `_sch_istable'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:5185:
undefined reference to `_sch_istable'
/lib/aarch64-linux-gnu/libbfd.a(elflink.o): In function
`bfd_elf_size_dynamic_sections':
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:6373:
undefined reference to `lbasename'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:6604:
undefined reference to `lbasename'
/lib/aarch64-linux-gnu/libbfd.a(elflink.o): In function `elf_output_implib':
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:11172:
undefined reference to `xmalloc'
/lib/aarch64-linux-gnu/libbfd.a(archures.o): In function `bfd_default_scan':
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/archures.c:1171:
undefined reference to `_sch_istable'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/archures.c:1171:
undefined reference to `_sch_istable'
...
/lib/aarch64-linux-gnu/libopcodes.a(arm-dis.o): In function `print_insn_neon':
/build/binutils-YyZpUv/binutils-2.28/builddir-single/opcodes/../../opcodes/arm-dis.c:4452:
undefined reference to `floatformat_ieee_single_little'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/opcodes/../../opcodes/arm-dis.c:4452:
undefined reference to `floatformat_ieee_single_little'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/opcodes/../../opcodes/arm-dis.c:4452:
undefined reference to `floatformat_to_double'
/lib/aarch64-linux-gnu/libopcodes.a(arm-dis.o): In function
`parse_disassembler_options':
/build/binutils-YyZpUv/binutils-2.28/builddir-single/opcodes/../../opcodes/arm-dis.c:6177:
undefined reference to `_sch_istable'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/opcodes/../../opcodes/arm-dis.c:6177:
undefined reference to `_sch_istable'
/build/binutils-YyZpUv/binutils-2.28/builddir-single/opcodes/../../opcodes/arm-dis.c:6180:
undefined reference to `_sch_istable'
/lib/aarch64-linux-gnu/libopcodes.a(arm-dis.o): In function `print_insn':
/build/binutils-YyZpUv/binutils-2.28/builddir-single/opcodes/../../opcodes/arm-dis.c:6754:
undefined reference to `_sch_istable'
collect2: error: ld returned 1 exit status
Makefile:96: recipe for target 'bpftool' failed
make: *** [bpftool] Error 1

> >
> > /lib/aarch64-linux-gnu/libbfd.a(elflink.o): In function
> > `elf_link_add_object_symbols':
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:4938:
> > undefined reference to `objalloc_free_block'
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:5185:
> > undefined reference to `_sch_istable'
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:5185:
> > undefined reference to `_sch_istable'
> > /lib/aarch64-linux-gnu/libbfd.a(elflink.o): In function
> > `bfd_elf_size_dynamic_sections':
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:6373:
> > undefined reference to `lbasename'
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:6604:
> > undefined reference to `lbasename'
> > /lib/aarch64-linux-gnu/libbfd.a(elflink.o): In function `elf_output_implib':
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/elflink.c:11172:
> > undefined reference to `xmalloc'
> > /lib/aarch64-linux-gnu/libbfd.a(archures.o): In function `bfd_default_scan':
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/archures.c:1171:
> > undefined reference to `_sch_istable'
> > /build/binutils-YyZpUv/binutils-2.28/builddir-single/bfd/../../bfd/archures.c:1171:
> > undefined reference to `_sch_istable'
> >
> > Binutils for arm64 are installed:
> >
> > $ dpkg -l | fgrep binutils
> > ii  binutils                           2.28-5
> > amd64        GNU assembler, linker and binary utilities
> > ii  binutils-aarch64-linux-gnu         2.28-5
> > amd64        GNU binary utilities, for aarch64-linux-gnu target
> > ii  binutils-dev                       2.28-5
> > amd64        GNU binary utilities (BFD development files)
> >
> > I'd love for it to work out of the box just like perf.