lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <935ac01a-8a1b-4986-9802-d2d1fd6445c2@geekplace.eu>
Date: Mon, 4 Nov 2024 13:52:35 +0100
From: Florian Schmaus <flo@...kplace.eu>
To: Masahiro Yamada <masahiroy@...nel.org>,
 Holger Hoffstätte <holger@...lied-asynchrony.com>
Cc: Nathan Chancellor <nathan@...nel.org>, Nicolas Schier
 <nicolas@...sle.eu>, Alexei Starovoitov <ast@...nel.org>,
 Daniel Borkmann <daniel@...earbox.net>, Andrii Nakryiko <andrii@...nel.org>,
 Martin KaFai Lau <martin.lau@...ux.dev>, Eduard Zingerman
 <eddyz87@...il.com>, Song Liu <song@...nel.org>,
 Yonghong Song <yonghong.song@...ux.dev>,
 John Fastabend <john.fastabend@...il.com>, KP Singh <kpsingh@...nel.org>,
 Stanislav Fomichev <sdf@...ichev.me>, Hao Luo <haoluo@...gle.com>,
 Jiri Olsa <jolsa@...nel.org>, bpf@...r.kernel.org,
 linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] kbuild,bpf: pass make jobs' value to pahole

On 03/11/2024 14.22, Masahiro Yamada wrote:
> On Sun, Nov 3, 2024 at 9:04 PM Holger Hoffstätte
> <holger@...lied-asynchrony.com> wrote:
>>
>> On 2024-11-02 11:04, Florian Schmaus wrote:
>>> Pass the value of make's -j/--jobs argument to pahole, to avoid out of
>>> memory errors and make pahole respect the "jobs" value of make.
>>>
>>> On systems with little memory but many cores, invoking pahole using -j
>>> without argument potentially creates too many pahole instances,
>>> causing an out-of-memory situation. Instead, we should pass make's
>>> "jobs" value as an argument to pahole's -j, which is likely configured
>>> to be (much) lower than the actual core count on such systems.
>>>
>>> If make was invoked without -j, either via cmdline or MAKEFLAGS, then
>>> JOBS will be simply empty, resulting in the existing behavior, as
>>> expected.
>>>
>>> Signed-off-by: Florian Schmaus <flo@...kplace.eu>
>>
>> As discussed on IRC:
> 
> Do not do this. Others do not see what was discussed.

Sorry, you are right. However, not much was discussed. Holger just 
pointed out that the memory usage of pahole was already reported as 
problematic in

https://lore.kernel.org/lkml/20240820085950.200358-1-jirislaby@kernel.org/

My patch would potentially help there as well, as it allows the user to 
limit the number of threads used by pahole.


> I guess the right thing to do is to join the jobserver.
> 
> https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html

Yes, this would be the ideal solution. Until it is implemented, the 
proposed patch is probably the next best thing.

- Florian

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ