linux-kernel - Re: [RFC] kbuild: bpf: Do not run pahole with -j on 32bit userspace

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAEf4Bza2oR1VLyGLyWRBbMbB00DQDFTCHNU_WT+0ktoWPk1Edw@mail.gmail.com>
Date: Mon, 26 Aug 2024 15:07:14 -0700
From: Andrii Nakryiko <andrii.nakryiko@...il.com>
To: Arnaldo Carvalho de Melo <acme@...nel.org>
Cc: Phil Auld <pauld@...hat.com>, Sedat Dilek <sedat.dilek@...il.com>, 
	Jiri Slaby <jirislaby@...nel.org>, Shung-Hsi Yu <shung-hsi.yu@...e.com>, dwarves@...r.kernel.org, 
	Jiri Olsa <olsajiri@...il.com>, masahiroy@...nel.org, linux-kernel@...r.kernel.org, 
	Nathan Chancellor <nathan@...nel.org>, Nicolas Schier <nicolas@...sle.eu>, 
	Alexei Starovoitov <ast@...nel.org>, Daniel Borkmann <daniel@...earbox.net>, 
	Andrii Nakryiko <andrii@...nel.org>, Martin KaFai Lau <martin.lau@...ux.dev>, 
	Eduard Zingerman <eddyz87@...il.com>, Song Liu <song@...nel.org>, 
	Yonghong Song <yonghong.song@...ux.dev>, John Fastabend <john.fastabend@...il.com>, 
	KP Singh <kpsingh@...nel.org>, Stanislav Fomichev <sdf@...ichev.me>, Hao Luo <haoluo@...gle.com>, 
	linux-kbuild@...r.kernel.org, bpf@...r.kernel.org, msuchanek@...e.com, 
	usamaarif642@...il.com
Subject: Re: [RFC] kbuild: bpf: Do not run pahole with -j on 32bit userspace

On Mon, Aug 26, 2024 at 1:04 PM Arnaldo Carvalho de Melo
<acme@...nel.org> wrote:
>
> On Mon, Aug 26, 2024 at 02:48:18PM -0400, Phil Auld wrote:
> > On Mon, Aug 26, 2024 at 08:42:10PM +0200 Sedat Dilek wrote:
> > > On Mon, Aug 26, 2024 at 7:03 PM Arnaldo Carvalho de Melo
> > > <acme@...nel.org> wrote:
> > > >
> > > > On Mon, Aug 26, 2024 at 10:57:22AM +0200, Jiri Slaby wrote:
> > > > > On 22. 08. 24, 17:24, Arnaldo Carvalho de Melo wrote:
> > > > > > On Thu, Aug 22, 2024 at 11:55:05AM +0800, Shung-Hsi Yu wrote:
> > > > > > I stumbled on this limitation as well when trying to build the kernel on
> > > > > > a Libre Computer rk3399-pc board with only 4GiB of RAM, there I just
> > > > > > created a swapfile and it managed to proceed, a bit slowly, but worked
> > > > > > as well.
> > > > >
> > > > > Here, it hits the VM space limit (3 G).
> > > >
> > > > right, in my case it was on a 64-bit system, so just not enough memory,
> > > > not address space.
> > > >
> > > > > > Please let me know if what is in the 'next' branch of:
> > > >
> > > > > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git
> > > >
> > > > > > Works for you, that will be extra motivation to move it to the master
> > > > > > branch and cut 1.28.
> > > >
> > > > > on 64bit (-j1):
> > > > > * master: 3.706 GB
> > > > > (* master + my changes: 3.559 GB)
> > > > > * next: 3.157 GB
> > > >
> > > > > on 32bit:
> > > > >  * master-j1: 2.445 GB
> > > > >  * master-j16: 2.608 GB
> > > > >  * master-j32: 2.811 GB
> > > > >  * next-j1: 2.256 GB
> > > > >  * next-j16: 2.401 GB
> > > > >  * next-j32: 2.613 GB
> > > > >
> > > > > It's definitely better. So I think it could work now, if the thread count
> > > > > was limited to 1 on 32bit. As building with -j10, -j20 randomly fails on
> > > > > random machines (32bit processes only of course). Unlike -j1.
> > > >
> > > > Cool, I just merged a patch from Alan Maguire that should help with the
> > > > parallel case, would be able to test it? It is in the 'next' branch:
> > > >
> > > > ⬢[acme@...lbox pahole]$ git log --oneline -5
> > > > f37212d1611673a2 (HEAD -> master) pahole: Teduce memory usage by smarter deleting of CUs
> > > >
> > >
> > > *R*edzce? memory usage ...
> > >
> >
> > If you meant that further typo it's golden, and if not the irony is rich :)
> >
> > Either way this is my favorite email of the day!
>
> Hahaha, I went to uppercase what comes after the colon and introduced
> that typo ;-)
>
> Faxing it....

typo-fest continues ;)

but it's great to see that a new pahole release is coming (cc'ing
Usama), we are eagerly waiting for one of the bug fixes that will go
into 1.28 (one of the first commits after 1.27 was cut). Note, libbpf
CI's for pahole-staging is failing, but that seems a one-off test and
has nothing to do with pahole being broken.

>
> - Arnaldo
>