linux-kernel - Re: [PATCH 1/1] kbuild: deb-pkg: Allow parallel build

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 15 Mar 2023 23:18:28 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     Bastian Germann <bage@...utronix.de>
Cc:     Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 1/1] kbuild: deb-pkg: Allow parallel build

On Tue, Mar 14, 2023 at 3:56 AM Bastian Germann <bage@...utronix.de> wrote:
>
> Am 13.03.23 um 18:59 schrieb Masahiro Yamada:
> > On Tue, Mar 14, 2023 at 2:10 AM Bastian Germann <bage@...utronix.de> wrote:
> >>
> >> Currently, the only way to build the deb-pkg generated package parallely
> >> is adding -jN to the MAKEFLAGS environment variable. The package ignores
> >> the usual parallel build option that is described in Debian Policy §4.9.1.
> >
> >
> >
> > "dpkg-buildpackage -b -j16" worked for me.
>
> This ends up in DEB_BUILD_OPTIONS=parallel=16 being set and the call:
> /usr/bin/make -f ./Makefile ARCH=x86    KERNELRELEASE=6.3.0-rc2 KBUILD_BUILD_VERSION=1  olddefconfig all

"dpkb-buildpackage -j<N>" sets not only DEB_BUILD_OPTIONS
but also MAKEFLAGS.

This is clearly explained in "man dpkb-buildpackage".

 -j, --jobs[=jobs|auto]
   ...
  Will add itself to the MAKEFLAGS environment variable, which should
  cause all subsequent make invocations to inherit the option, thus
  forcing the parallel setting on the packaging ...

Your statement sounds like

 'MAKEFLAGS=-j<N> dpkg-buildpackage'

is the only way to build packages in parallel.

Apparently, dpkg-buildpackage provides a much shorter way
and invokes internal Make in parallel.

>
> So it is not used to run the actual build, just the top level `make -f debian/rules` invocation.
> You can set --jobs-force=16, which ends up in MAKEFLAGS but the point of the patch is that one can use
> the usual way of parallelizing.

What is the "usual" way in this context?

Do you mean
'DEB_BUILD_OPTIONS=parallel=16 dpkg-buildpackage -b'
is the usual way for parallel building?

If so, I agree. This patch caters to this case.

But, I think your code should go to debian/rules
instead of scripts/package/deb-build-option.
Kbuild's "make clean" works much faster with the parallel option.

Also, the commit description should explain the
benefit of this patch.

>
> Side note: Without the patch, the build is run with -j1 regardless of being called from the Linux Makefile
> or dpkg-buildpackage.

Try:

 $ make deb-pkg -j16

and

 $ dpkg-buildpackage -b -j16

Both run the package building in parallel.

You see a big difference in build speed w/wo the -j option.
(and the build logs are interleaved, since they
are emitted by multiple threads)

--
Best Regards
Masahiro Yamada