linux-kernel - Re: [PATCH v5] kbuild: add variables for compression tools

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <d7e4a235-35eb-1c13-09dc-19a523cbf4ea@linux.com>
Date:   Mon, 8 Jun 2020 13:28:07 +0300
From:   Denis Efremov <efremov@...ux.com>
To:     Masahiro Yamada <masahiroy@...nel.org>,
        Guenter Roeck <linux@...ck-us.net>
Cc:     Stephen Rothwell <sfr@...b.auug.org.au>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v5] kbuild: add variables for compression tools



On 6/8/20 7:59 AM, Masahiro Yamada wrote:
> On Mon, Jun 8, 2020 at 10:30 AM Guenter Roeck <linux@...ck-us.net> wrote:
>>
>> Hi,
>>
>> On Fri, Jun 05, 2020 at 10:39:55AM +0300, Denis Efremov wrote:
>>> Allow user to use alternative implementations of compression tools,
>>> such as pigz, pbzip2, pxz. For example, multi-threaded tools to
>>> speed up the build:
>>> $ make GZIP=pigz BZIP2=pbzip2
>>>
>>> Variables _GZIP, _BZIP2, _LZOP are used internally because original env
>>> vars are reserved by the tools. The use of GZIP in gzip tool is obsolete
>>> since 2015. However, alternative implementations (e.g., pigz) still rely
>>> on it. BZIP2, BZIP, LZOP vars are not obsolescent.
>>>
>>
>> When building mips:defconfig, this patch results in:
>>
>> Building mips:defconfig ... failed
>> --------------
>> Error log:
>> /bin/sh: -n: command not found
>> make[3]: *** [kernel/config_data.gz] Error 127
>> make[3]: *** Deleting file 'kernel/config_data.gz'
>> make[3]: *** Waiting for unfinished jobs....
>> make[2]: *** [kernel] Error 2
>> make[2]: *** Waiting for unfinished jobs....
>> make[1]: *** [autoksyms_recursive] Error 2
>> make: *** [__sub-make] Error 2
>>
>> Reverting this patch fixes the problem. Bisect log is attached.
>>
>> Guenter
>

I tried to reproduce it with cross-compilation on Fedora32.
$ export ARCH=mips
$ export CROSS_COMPILE=mips64-linux-gnu-
$ make defconfig
$ make -j12

And the kernel builds successfully. Could you please provide details about your
compilation steps and environment, esp. what "env | grep ZIP" shows,
"gzip --version", "sh --version", "bash --version"? This will be very helpful.

Additionally:
$ make GZIP=gzip -j12 # works
$ make GZIP=pigz -j12 # works
$ make GZIP=nosuchcommand -j12 # fails, as expected
 
> 
> Agh, this is because of CONFIG_TRIM_UNUSED_KSYMS.
> 

Hmm, it somehow works on my machine. But yes, this call looks like a problem
for these env vars:

autoksyms_recursive: descend modules.order                                       
        $(Q)$(CONFIG_SHELL) $(srctree)/scripts/adjust_autoksyms.sh \             
          "$(MAKE) -f $(srctree)/Makefile vmlinux" 

> Also, the distro package builds are broken
> e.g.  make GZIP=gzip bindeb-pkg
> 

Yes, thanks.

> Denis,
> 
> I think we should go back to the original
> KGZIP, KBZIP2, KLZOP.
> 

Given that the original patch is already in the Linus tree,
I've sent a hotfix. Commit message is not perfect, because I
didn't have enough time to deeply debug it. I just hope that
the original patch didn't broke too many builds. Maybe later
I will try to prepare a patch with GZIP again when I will fully
debug these corner cases.

Thanks,
Denis