Date:   Wed, 26 Jul 2023 21:20:33 -0700
From:   Ian Rogers <irogers@...gle.com>
To:     Bagas Sanjaya <bagasdotme@...il.com>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Adrian Hunter <adrian.hunter@...el.com>,
        Kajol Jain <kjain@...ux.ibm.com>,
        Kan Liang <kan.liang@...ux.intel.com>,
        John Garry <john.g.garry@...cle.com>,
        Marcus Seyfarth <m.seyfarth@...il.com>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Linux Regressions <regressions@...ts.linux.dev>,
        Linux Perf <linux-perf-users@...r.kernel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>
Subject: Re: [perf] Build error when using LTO on 6.4

On Wed, Jul 26, 2023 at 8:55 PM Bagas Sanjaya <bagasdotme@...il.com> wrote:
>
> Hi,
>
> I notice a build regression on Bugzilla [1]. Quoting from it:
>
> > Starting with Kernel 6.4, I see a build error when using "-flto" to build perf. This used to work fine with earlier Kernels when using either bfd or mold as linker when using GCC 13.1.1.

Hmm.. strange I've never seen this work with GCC.

> >
> > This is the PKGBUILD I use as my build recipe: https://github.com/ms178/archpkgbuilds/blob/main/packages/bpf/PKGBUILD

If this binary is for distribution BUILD_BPF_SKEL=1 should be set. We
need to make this an opt-out rather than opt-in. The last time we did
this there was some friction:
https://lore.kernel.org/lkml/CAHk-=wjY_3cBELRSLMpqCt6Eb71Qei2agfKSNsrr5KcpdEQCaA@mail.gmail.com/

> >
> > This is the konsole output I get now with Kernel 6.4.5:
> >
> > [CODE]
> > CFLAGS='-Wno-error=bad-function-cast -Wno-error=declaration-after-statement -Wno-error=switch-enum -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -Wno-type-limits -Wstrict-aliasing=3 -Wshadow -DHAVE_SYSCALL_TABLE_SUPPORT -DHAVE_ARCH_X86_64_SUPPORT -Iarch/x86/include/generated -DHAVE_PERF_REGS_SUPPORT -DHAVE_ARCH_REGS_QUERY_REGISTER_OFFSET -Werror -DNDEBUG=1 -O6 -fno-omit-frame-pointer -ggdb3 -funwind-tables -Wall -Wextra -std=gnu11 -fstack-protector-all -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/tmp/makepkg/linux-tools/src/linux/tools/perf/util/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/arch/x86/include -I/tmp/makepkg/linux-tools/src/linux/tools/include/ -I/tmp/makepkg/linux-tools/src/linux/tools/arch/x86/include/uapi -I/tmp/makepkg/linux-tools/src/linux/tools/include/uapi -I/tmp/makepkg/linux-tools/src/linux/tools/arch/x86/include/ -I/tmp/makepkg/linux-tools/src/linux/tools/arch/x86/ -I/tmp/makepkg/linux-tools/src/linux/tools/perf/util -I/tmp/makepkg/linux-tools/src/linux/tools/perf -DHAVE_PTHREAD_ATTR_SETAFFINITY_NP -DHAVE_PTHREAD_BARRIER -DHAVE_EVENTFD_SUPPORT -DHAVE_GET_CURRENT_DIR_NAME -DHAVE_GETTID -DHAVE_FILE_HANDLE -DHAVE_DWARF_GETLOCATIONS_SUPPORT -DHAVE_AIO_SUPPORT -DHAVE_SCANDIRAT_SUPPORT -DHAVE_SCHED_GETCPU_SUPPORT -DHAVE_SETNS_SUPPORT -DHAVE_LIBELF_SUPPORT -DHAVE_ELF_GETPHDRNUM_SUPPORT -DHAVE_GELF_GETNOTE_SUPPORT -DHAVE_ELF_GETSHDRSTRNDX_SUPPORT -DHAVE_DEBUGINFOD_SUPPORT -DHAVE_DWARF_SUPPORT  -DHAVE_LIBBPF_SUPPORT -DHAVE_BPF_PROLOGUE -DHAVE_JITDUMP -DHAVE_DWARF_UNWIND_SUPPORT -DNO_LIBUNWIND_DEBUG_FRAME -DHAVE_LIBUNWIND_SUPPORT -DHAVE_LIBCRYPTO_SUPPORT -DHAVE_SLANG_SUPPORT -DNO_LIBPERL -DHAVE_TIMERFD_SUPPORT -DHAVE_LIBPYTHON_SUPPORT -DHAVE_CXA_DEMANGLE_SUPPORT -DHAVE_ZLIB_SUPPORT -DHAVE_LZMA_SUPPORT -DHAVE_ZSTD_SUPPORT -DHAVE_LIBCAP_SUPPORT -DHAVE_BACKTRACE_SUPPORT -DHAVE_LIBNUMA_SUPPORT -DHAVE_KVM_STAT_SUPPORT -DDISASM_FOUR_ARGS_SIGNATURE -DDISASM_INIT_STYLED -DHAVE_PERF_READ_VDSO32 -DHAVE_AUXTRACE_SUPPORT -DHAVE_LIBTRACEEVENT -DLIBTRACEEVENT_VERSION=66813 -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libapi/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libbpf/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libsubcmd/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libsymbol/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libperf/include' LDFLAGS='-Wl,-O3,--as-needed,-Bsymbolic-functions,--sort-common,-flto=auto -fopenmp -fuse-ld=mold -march=native -mtune=native -maes -mbmi2 -mpclmul -Wl,-zmax-page-size=0x200000 -Wl,-z,noexecstack  -lunwind-x86_64 -lunwind -llzma  -L/usr/lib ' \
> >   '/usr/bin/python' util/setup.py \
> >   --quiet build_ext; \
> > cp python_ext_build/lib/perf*.so python/
> > rm -f -f /tmp/makepkg/linux-tools/src/linux/tools/perf/libbpf/libbpf.a; ar rcs /tmp/makepkg/linux-tools/src/linux/tools/perf/libbpf/libbpf.a /tmp/makepkg/linux-tools/src/linux/tools/perf/libbpf/staticobjs/libbpf-in.o
> > [snip]
> > gcc -Wno-error=bad-function-cast -Wno-error=declaration-after-statement -Wno-error=switch-enum -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -Wno-type-limits -Wstrict-aliasing=3 -Wshadow -DHAVE_SYSCALL_TABLE_SUPPORT -DHAVE_ARCH_X86_64_SUPPORT -Iarch/x86/include/generated -DHAVE_PERF_REGS_SUPPORT -DHAVE_ARCH_REGS_QUERY_REGISTER_OFFSET -Werror -DNDEBUG=1 -O6 -fno-omit-frame-pointer -ggdb3 -funwind-tables -Wall -Wextra -std=gnu11 -fstack-protector-all -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I/tmp/makepkg/linux-tools/src/linux/tools/perf/util/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/arch/x86/include -I/tmp/makepkg/linux-tools/src/linux/tools/include/ -I/tmp/makepkg/linux-tools/src/linux/tools/arch/x86/include/uapi -I/tmp/makepkg/linux-tools/src/linux/tools/include/uapi -I/tmp/makepkg/linux-tools/src/linux/tools/arch/x86/include/ -I/tmp/makepkg/linux-tools/src/linux/tools/arch/x86/ -I/tmp/makepkg/linux-tools/src/linux/tools/perf/util -I/tmp/makepkg/linux-tools/src/linux/tools/perf -DHAVE_PTHREAD_ATTR_SETAFFINITY_NP -DHAVE_PTHREAD_BARRIER -DHAVE_EVENTFD_SUPPORT -DHAVE_GET_CURRENT_DIR_NAME -DHAVE_GETTID -DHAVE_FILE_HANDLE -DHAVE_DWARF_GETLOCATIONS_SUPPORT -DHAVE_AIO_SUPPORT -DHAVE_SCANDIRAT_SUPPORT -DHAVE_SCHED_GETCPU_SUPPORT -DHAVE_SETNS_SUPPORT -DHAVE_LIBELF_SUPPORT -DHAVE_ELF_GETPHDRNUM_SUPPORT -DHAVE_GELF_GETNOTE_SUPPORT -DHAVE_ELF_GETSHDRSTRNDX_SUPPORT -DHAVE_DEBUGINFOD_SUPPORT -DHAVE_DWARF_SUPPORT  -DHAVE_LIBBPF_SUPPORT -DHAVE_BPF_PROLOGUE -DHAVE_JITDUMP -DHAVE_DWARF_UNWIND_SUPPORT -DNO_LIBUNWIND_DEBUG_FRAME -DHAVE_LIBUNWIND_SUPPORT -DHAVE_LIBCRYPTO_SUPPORT -DHAVE_SLANG_SUPPORT -DNO_LIBPERL -DHAVE_TIMERFD_SUPPORT -DHAVE_LIBPYTHON_SUPPORT -DHAVE_CXA_DEMANGLE_SUPPORT -DHAVE_ZLIB_SUPPORT -DHAVE_LZMA_SUPPORT -DHAVE_ZSTD_SUPPORT -DHAVE_LIBCAP_SUPPORT -DHAVE_BACKTRACE_SUPPORT -DHAVE_LIBNUMA_SUPPORT -DHAVE_KVM_STAT_SUPPORT -DDISASM_FOUR_ARGS_SIGNATURE -DDISASM_INIT_STYLED -DHAVE_PERF_READ_VDSO32 -DHAVE_AUXTRACE_SUPPORT -DHAVE_LIBTRACEEVENT -DLIBTRACEEVENT_VERSION=66813 -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libapi/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libbpf/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libsubcmd/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libsymbol/include -I/tmp/makepkg/linux-tools/src/linux/tools/perf/libperf/include -Wl,-O3,--as-needed,-Bsymbolic-functions,--sort-common,-flto=auto -fopenmp -fuse-ld=mold -march=native -mtune=native -maes -mbmi2 -mpclmul -Wl,-zmax-page-size=0x200000 -Wl,-z,noexecstack  -lunwind-x86_64 -lunwind -llzma  -L/usr/lib  \
> >         perf-in.o pmu-events/pmu-events-in.o -Wl,--whole-archive /tmp/makepkg/linux-tools/src/linux/tools/perf/libapi/libapi.a /tmp/makepkg/linux-tools/src/linux/tools/perf/libperf/libperf.a /tmp/makepkg/linux-tools/src/linux/tools/perf/libsubcmd/libsubcmd.a /tmp/makepkg/linux-tools/src/linux/tools/perf/libsymbol/libsymbol.a /tmp/makepkg/linux-tools/src/linux/tools/perf/libbpf/libbpf.a  -Wl,--no-whole-archive -Wl,--start-group -lpthread -lrt -lm -ldl -lelf -ldebuginfod -ldw -lunwind-x86_64 -lunwind -llzma -lcrypto -lslang -lpython3.11 -ldl -lm -lutil -lstdc++ -lz -llzma -lzstd -lcap -lnuma -ltraceevent -Wl,--end-group -o perf
> > mold: error: undefined symbol: main
>
> >>>> referenced by /usr/lib/Scrt1.o:(.text)
> >>>>               /usr/lib/Scrt1.o:(_start)
> > mold: error: undefined symbol: perf_pmu__getcpuid
> >>>> referenced by pmu-events.c
> >>>>               pmu-events/pmu-events-in.o:(perf_pmu__find_events_table)>>>
> >>>>               referenced by pmu-events.c
> >>>>               pmu-events/pmu-events-in.o:(perf_pmu__find_metrics_table)
> > mold: error: undefined symbol: strcmp_cpuid_str
> >>>> referenced by pmu-events.c
> >>>>               pmu-events/pmu-events-in.o:(perf_pmu__find_events_table)>>>
> >>>>               referenced by pmu-events.c
> >>>>               pmu-events/pmu-events-in.o:(perf_pmu__find_metrics_table)>>>
> >>>>               referenced by pmu-events.c
> >>>>               pmu-events/pmu-events-in.o:(find_core_events_table)>>>
> >>>>               referenced 1 more times
> >
> > mold: error: undefined symbol: _ctype
> >>>> referenced by kallsyms.c
> >>>>
> >>>>               /tmp/makepkg/linux-tools/src/linux/tools/perf/libsymbol/libsymbol.a(libsymbol-in.o):(__tolower)>>>
> >>>>               referenced by kallsyms.c
> >>>>
> >>>>               /tmp/makepkg/linux-tools/src/linux/tools/perf/libsymbol/libsymbol.a(libsymbol-in.o):(__toupper)
> > mold: error: undefined symbol: strlcpy
> >>>> referenced by exec-cmd.c
> >>>>
> >>>>               /tmp/makepkg/linux-tools/src/linux/tools/perf/libsubcmd/libsubcmd.a(libsubcmd-in.o):(get_pwd_cwd)
> > [/CODE]
> >
> > I have noticed a patch series of Ian Rodgers that might adress this issue, but his reported errors were different to mine, see: https://www.spinics.net/lists/kernel/msg4872566.html
> >
> > Compiler: gcc-version 13.1.1 20230714 (GCC)
>
> See Bugzilla for the full thread.
>
> Anyway, I'm adding this regression to regzbot:
>
> #regzbot introduced: v6.3..v6.4.5 https://bugzilla.kernel.org/show_bug.cgi?id=217706

Thanks, I'm not sure there is anything actionable for the bug.
clang/llvm has always worked with LTO at Google. With my changes gcc
works for me with some non-zero effort.

Thanks,
Ian

> Thanks.
>
> [1]: https://bugzilla.kernel.org/show_bug.cgi?id=217706
>
> --
> An old man doll... just what I always wanted! - Clara

Powered by blists - more mailing lists