linux-kernel - Re: [PATCH 2/4] kbuild: avoid too many execution of scripts/pahole-flags.sh

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNARRtCQOoWT0q5EjJhoMrfAVbCg_fREeA+NpUHBRbB5oww@mail.gmail.com>
Date:   Wed, 18 Oct 2023 14:00:42 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     Andrii Nakryiko <andrii.nakryiko@...il.com>
Cc:     linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        Alex Gaynor <alex.gaynor@...il.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Alice Ryhl <aliceryhl@...gle.com>,
        Andreas Hindborg <a.hindborg@...sung.com>,
        Andrii Nakryiko <andrii@...nel.org>,
        Benno Lossin <benno.lossin@...ton.me>,
        Björn Roy Baron <bjorn3_gh@...tonmail.com>,
        Boqun Feng <boqun.feng@...il.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Gary Guo <gary@...yguo.net>, Hao Luo <haoluo@...gle.com>,
        Jiri Olsa <jolsa@...nel.org>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...nel.org>,
        Martin KaFai Lau <martin.lau@...ux.dev>,
        Miguel Ojeda <ojeda@...nel.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>, Song Liu <song@...nel.org>,
        Stanislav Fomichev <sdf@...gle.com>,
        Wedson Almeida Filho <wedsonaf@...il.com>,
        Yonghong Song <yonghong.song@...ux.dev>, bpf@...r.kernel.org,
        rust-for-linux@...r.kernel.org
Subject: Re: [PATCH 2/4] kbuild: avoid too many execution of scripts/pahole-flags.sh

On Wed, Oct 18, 2023 at 4:57 AM Andrii Nakryiko
<andrii.nakryiko@...il.com> wrote:
>
> On Tue, Oct 17, 2023 at 3:38 AM Masahiro Yamada <masahiroy@...nel.org> wrote:
> >
> > scripts/pahole-flags.sh is executed so many times.
> >
> > You can check how many times it is invoked during the build, as follows:
> >
> >   $ cat <<EOF >> scripts/pahole-flags.sh
> >   > echo "scripts/pahole-flags.sh was executed" >&2
> >   > EOF
> >
> >   $ make -s
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >   scripts/pahole-flags.sh was executed
> >     [ lots of repeated lines suppressed... ]
> >
> > This scripts is exectuted more than 20 times during the kernel build
> > because PAHOLE_FLAGS is a recursively expanded variable and exported
> > to sub-processes.
> >
> > With the GNU Make >= 4.4, it is executed more than 60 times because
> > exported variables are also passed to other $(shell ) invocations.
> > Without careful coding, it is known to cause an exponential fork
> > explosion. [1]
> >
> > The use of $(shell ) in an exported recursive variable is likely wrong
> > because $(shell ) is always evaluated due to the 'export' keyword, and
> > the evaluation can occur multiple times by the nature of recursive
> > variables.
> >
> > Convert the shell script to a Makefile, which is included only when
> > CONFIG_DEBUG_INFO_BTF=y.
> >
> > [1]: https://savannah.gnu.org/bugs/index.php?64746
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> > ---
> >
> >  Makefile                |  4 +---
> >  scripts/Makefile.btf    | 19 +++++++++++++++++++
> >  scripts/pahole-flags.sh | 30 ------------------------------
> >  3 files changed, 20 insertions(+), 33 deletions(-)
> >  create mode 100644 scripts/Makefile.btf
> >  delete mode 100755 scripts/pahole-flags.sh
> >
> > diff --git a/Makefile b/Makefile
> > index fed9a6cc3665..eaddec67e5e1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -513,8 +513,6 @@ LZ4         = lz4c
> >  XZ             = xz
> >  ZSTD           = zstd
> >
> > -PAHOLE_FLAGS   = $(shell PAHOLE=$(PAHOLE) $(srctree)/scripts/pahole-flags.sh)
>
> What if we just used := here? Wouldn't it avoid unnecessary multiple executions?


Yeah, := is less silly than =.


But, I do not like to run the script for non-build targets
such as 'make clean', 'make help', etc.

Also, when building with CONFIG_DEBUG_INFO_BTF=n,
the shell is forked to compute PAHOLE_FLAGS,
which we know are unnecessary.





--
Best Regards

Masahiro Yamada