linux-kernel - Re: [RFC] kbuild: bpf: Do not run pahole with -j on 32bit userspace

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <523c1afa-ed9d-4c76-baea-1c43b1b0c682@kernel.org>
Date: Wed, 21 Aug 2024 07:32:21 +0200
From: Jiri Slaby <jirislaby@...nel.org>
To: Jiri Olsa <olsajiri@...il.com>
Cc: masahiroy@...nel.org, linux-kernel@...r.kernel.org,
 Jiri Slaby <jslaby@...e.cz>, Nathan Chancellor <nathan@...nel.org>,
 Nicolas Schier <nicolas@...sle.eu>, Alexei Starovoitov <ast@...nel.org>,
 Daniel Borkmann <daniel@...earbox.net>, Andrii Nakryiko <andrii@...nel.org>,
 Martin KaFai Lau <martin.lau@...ux.dev>, Eduard Zingerman
 <eddyz87@...il.com>, Song Liu <song@...nel.org>,
 Yonghong Song <yonghong.song@...ux.dev>,
 John Fastabend <john.fastabend@...il.com>, KP Singh <kpsingh@...nel.org>,
 Stanislav Fomichev <sdf@...ichev.me>, Hao Luo <haoluo@...gle.com>,
 linux-kbuild@...r.kernel.org, bpf@...r.kernel.org, shung-hsi.yu@...e.com,
 msuchanek@...e.com
Subject: Re: [RFC] kbuild: bpf: Do not run pahole with -j on 32bit userspace

On 20. 08. 24, 16:33, Jiri Olsa wrote:
> On Tue, Aug 20, 2024 at 10:59:50AM +0200, Jiri Slaby (SUSE) wrote:
>> From: Jiri Slaby <jslaby@...e.cz>
>>
>> == WARNING ==
>> This is only a PoC. There are deficiencies like CROSS_COMPILE or LLVM
>> are completely unhandled.
>>
>> The simple version is just do there:
>>    ifeq ($(CONFIG_64BIT,y)
>> but it has its own deficiencies, of course.
>>
>> So any ideas, inputs?
>> == WARNING ==
>>
>> When pahole is run with -j on 32bit userspace (32bit pahole in
>> particular), it randomly fails with OOM:
>>> btf_encoder__tag_kfuncs: Failed to get ELF section(62) data: out of memory.
>>> btf_encoder__encode: failed to tag kfuncs!
>>
>> or simply SIGSEGV (failed to allocate the btf encoder).
>>
>> It very depends on how many threads are created.
>>
>> So do not invoke pahole with -j on 32bit.
> 
> could you share more details about your setup?
> 
> does it need to run on pure 32bit to reproduce?

armv7l builds are 32bit only.

> I can't reproduce when
> doing cross build and running 32 bit pahole on x86_64..

i586 is built using 64bit kernel. It is enough to have 32bit userspace.
As written in the linked bug:
https://bugzilla.suse.com/show_bug.cgi?id=1229450#c6

FWIW, steps to reproduce locally:
docker pull jirislaby/pahole_crash
docker run -it jirislaby/pahole_crash

The VM space of pahole is exhausted:
process map: https://bugzilla.suse.com/attachment.cgi?id=876821
strace of mmaps: https://bugzilla.suse.com/attachment.cgi?id=876822

You need to run with large enough -j on a fast machine. Note that this 
happens on build hosts even with -j4, but they are under heavy load, so 
parallelism of held memory is high.

On my box with 16 cores, it is (likely far) enough to run with -j32.

> I do see some
> errors though
> 
>    [667939] STRUCT bpf_prog_aux Error emitting BTF type
>    Encountered error while encoding BTF.

It's possible that it is one of the errors. There are different ones. As 
I wrote above, sometimes it is a crash, sometimes it is the failure I 
mentioned above. But it always ends up with a failed build:
 > libbpf: failed to find '.BTF' ELF section in vmlinux
 > FAILED: load BTF from vmlinux: No data available
 > make[2]: *** [../scripts/Makefile.vmlinux:34: vmlinux] Error 255
 > make[2]: *** Deleting file 'vmlinux'
 > make[1]: *** 
[/home/abuild/rpmbuild/BUILD/kernel-vanilla-6.11~rc3.338.gc3f2d783a459/linux-6.11-rc3-338-gc3f2d783a459/Makefile:1158: 
vmlinux] Error 2
 > make: *** [../Makefile:224: __sub-make] Error 2
 > error: Bad exit status from /var/tmp/rpm-tmp.olf5Nu (%build)

thanks,
-- 
js
suse labs