linux-kernel - Re: [PATCH v3] docs: Use make invocation's -j argument for parallelism

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 4 Oct 2019 12:58:49 +0200
From:   Christian Borntraeger <borntraeger@...ibm.com>
To:     Kees Cook <keescook@...omium.org>, Jonathan Corbet <corbet@....net>
Cc:     Mauro Carvalho Chehab <mchehab+samsung@...nel.org>,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
        Heiko Carstens <heiko.carstens@...ibm.com>,
        Vasily Gorbik <gor@...ux.ibm.com>,
        Linux-Next Mailing List <linux-next@...r.kernel.org>,
        Stephen Rothwell <sfr@...b.auug.org.au>
Subject: Re: [PATCH v3] docs: Use make invocation's -j argument for
 parallelism



On 04.10.19 10:04, Christian Borntraeger wrote:
> 
> On 25.09.19 01:29, Kees Cook wrote:
>> While sphinx 1.7 and later supports "-jauto" for parallelism, this
>> effectively ignores the "-j" flag used in the "make" invocation, which
>> may cause confusion for build systems. Instead, extract the available
>> parallelism from "make"'s job server (since it is not exposed in any
>> special variables) and use that for the "sphinx-build" run. Now things
>> work correctly for builds where -j is specified at the top-level:
>>
>> 	make -j16 htmldocs
>>
>> If -j is not specified, continue to fallback to "-jauto" if available.
>>
>> Signed-off-by: Kees Cook <keescook@...omium.org>
>> ---
>> v3: python2, specific exceptions, correct SPDX, blocking writer
>> v2: retain "-jauto" default behavior with top-level -j is missing.              
> [...]
>> diff --git a/scripts/jobserver-count b/scripts/jobserver-count
>> new file mode 100755
>> index 000000000000..0b482d6884d2
>> --- /dev/null
>> +++ b/scripts/jobserver-count
>> @@ -0,0 +1,58 @@
>> +#!/usr/bin/env python
> 
> 
> This breaks our daily linux-next build for an fedora 30 rpm on s390x:
> 
> + /usr/lib/rpm/redhat/brp-mangle-shebangs
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/profile2linkerlist.pl from /usr/bin/env perl to #!/usr/bin/perl
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/headerdep.pl from /usr/bin/env perl to #!/usr/bin/perl
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/package/buildtar from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/package/builddeb from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/package/mkspec from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/package/mkdebian from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/checksyscalls.sh from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/gen_ksymdeps.sh from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/makelst from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/checkversion.pl from /usr/bin/env perl to #!/usr/bin/perl
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/gcc-plugin.sh from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/gfp-translate from /bin/bash to #!/usr/bin/bash
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/tags.sh from /bin/bash to #!/usr/bin/bash
> *** ERROR: ambiguous python shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/jobserver-count: #!/usr/bin/env python. Change it to python3 (or python2) explicitly.
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/adjust_autoksyms.sh from /bin/sh to #!/usr/bin/sh
> mangling shebang in /usr/src/kernels/5.4.0-20191004.rc1.git155.311ef88adfa3.301.fc30.s390x+next/scripts/kernel-doc from /usr/bin/env perl to #!/usr/bin/perl
> [...]


Ok, adding something like 

+pathfix.py -pni "%{__python3} %{py3_shbang_opts}" scripts/jobserver-count

to the spec file fixed the problem. 

Question is, if we want to make the python version more specific or not.