linux-kernel - Re: [RFC] kbuild: bpf: Do not run pahole with -j on 32bit userspace

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZszfX0V1YgM0225K@x1>
Date: Mon, 26 Aug 2024 17:02:39 -0300
From: Arnaldo Carvalho de Melo <acme@...nel.org>
To: Jiri Slaby <jirislaby@...nel.org>
Cc: Shung-Hsi Yu <shung-hsi.yu@...e.com>, dwarves@...r.kernel.org,
	Jiri Olsa <olsajiri@...il.com>, masahiroy@...nel.org,
	linux-kernel@...r.kernel.org, Nathan Chancellor <nathan@...nel.org>,
	Nicolas Schier <nicolas@...sle.eu>,
	Alexei Starovoitov <ast@...nel.org>,
	Daniel Borkmann <daniel@...earbox.net>,
	Andrii Nakryiko <andrii@...nel.org>,
	Martin KaFai Lau <martin.lau@...ux.dev>,
	Eduard Zingerman <eddyz87@...il.com>, Song Liu <song@...nel.org>,
	Yonghong Song <yonghong.song@...ux.dev>,
	John Fastabend <john.fastabend@...il.com>,
	KP Singh <kpsingh@...nel.org>, Stanislav Fomichev <sdf@...ichev.me>,
	Hao Luo <haoluo@...gle.com>, linux-kbuild@...r.kernel.org,
	bpf@...r.kernel.org, msuchanek@...e.com
Subject: Re: [RFC] kbuild: bpf: Do not run pahole with -j on 32bit userspace

On Mon, Aug 26, 2024 at 10:57:22AM +0200, Jiri Slaby wrote:
> On 22. 08. 24, 17:24, Arnaldo Carvalho de Melo wrote:
> > On Thu, Aug 22, 2024 at 11:55:05AM +0800, Shung-Hsi Yu wrote:
> > I stumbled on this limitation as well when trying to build the kernel on
> > a Libre Computer rk3399-pc board with only 4GiB of RAM, there I just
> > created a swapfile and it managed to proceed, a bit slowly, but worked
> > as well.
> 
> Here, it hits the VM space limit (3 G).
> 
> > Please let me know if what is in the 'next' branch of:
> > 
> > https://git.kernel.org/pub/scm/devel/pahole/pahole.git
> > 
> > Works for you, that will be extra motivation to move it to the master
> > branch and cut 1.28.
> 
> on 64bit (-j1):
> * master: 3.706 GB
> (* master + my changes: 3.559 GB)
> * next: 3.157 GB
> 
> 
> on 32bit:
>  * master-j1: 2.445 GB
>  * master-j16: 2.608 GB
>  * master-j32: 2.811 GB
>  * next-j1: 2.256 GB
>  * next-j16: 2.401 GB
>  * next-j32: 2.613 GB
> 
> It's definitely better. So I think it could work now, if the thread count
> was limited to 1 on 32bit. As building with -j10, -j20 randomly fails on
> random machines (32bit processes only of course). Unlike -j1.

Great! I'm now processing a patch by Alan Maguire that should help with
parallel loading, I'll add it o the next branch, together with his work
on distilled BTF.

If you could test it, it would be great to see how much it helps with
teh serial case and if it allows for at least some parallel processing
on 32-bit architectures.

Thanks for all your effort on this, appreciated.

- Arnaldo