linux-kernel - Re: [PATCH 7/7] kbuild: link vmlinux just once for CONFIG_TRIM_UNUSED

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNATANL6Lfa6=05g9fCfj0ZLNHu_XnPEfurrOBQ8YXqUc4w@mail.gmail.com>
Date:   Thu, 15 Mar 2018 17:00:26 +0900
From:   Masahiro Yamada <yamada.masahiro@...ionext.com>
To:     Nicolas Pitre <nicolas.pitre@...aro.org>
Cc:     Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Jarod Wilson <jarod@...hat.com>,
        Prarit Bhargava <prarit@...hat.com>,
        Michal Marek <michal.lkml@...kovi.net>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 7/7] kbuild: link vmlinux just once for CONFIG_TRIM_UNUSED_KSYMS

2018-03-15 4:06 GMT+09:00 Nicolas Pitre <nicolas.pitre@...aro.org>:
> On Thu, 15 Mar 2018, Masahiro Yamada wrote:
>
>> If CONFIG_TRIM_UNUSED_KSYMS is enabled and the kernel is built from
>> a pristine state, the vmlinux is linked twice.
>>
>> [1] A user runs "make"
>>
>> [2] First build with empty autoksyms.h
>>
>> [3] adjust_autoksyms.sh updates autoksyms.h and recurses "make vmlinux"
>>
>>   --------(begin sub-make)--------
>>   [4] Second build with new autoksyms.h
>>
>>   [5] link-vmlinux.sh is invoked because "vmlinux" is missing
>>   ---------(end sub-make)---------
>>
>> [6] link-vmlinux.sh is invoked again despite "vmlinux" is up-to-date.
>>
>> The reason of [6] is probably because Make already decided to update
>> "vmlinux" at the time of [2] because "vmlinux" was missing when Make
>> generated the dependency list.
>
> But the dependency list for vmlinux contains FORCE so the target is
> always remade in (2) anyway. The decision to actually invoke
> link-vmlinux.sh comes from "if_changed". Why is $(call if_changed,...)
> in (6) not noticing that vmlinux is up to date?

if_changed (more specifically 'any-prereq')
is implemented based on '$?'.

So, this problem can be narrowed down to
how Make decides '$?'.


If you apply the first 6 patches, and put the following
debug code, you will see what has happened to $?


@@ -1024,6 +1026,7 @@ cmd_link-vmlinux =
                  \
        $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true)

 vmlinux: scripts/link-vmlinux.sh vmlinux_prereq $(vmlinux-deps) FORCE
+       @echo newer deps: $?
        +$(call if_changed,link-vmlinux)

 # Build samples along the rest of the kernel



I am not familiar with Make internal, but
I can test it with simple code.


[Test Makefile]
A: B
        cp B A
        echo $?

B: C
        cp C B
        touch A

[Result]
$ rm -rf A B C
$ touch C
$ make
cp C B
touch A
cp B A
echo B
B


In the recipe of 'B', 'A' is touched,
so the dependency 'A: B' has already been met
before the recipe of 'A' is executed,
but Make does not notice that fact,
then tries to update 'A'.


The situation is the same.
vmlinux has been updated in the recursed sub-make,
but it is not noticed.




>>
>> link-vmlinus.sh is costly, so it is better to not run it when unneeded.
>> Split CONFIG_TRIM_UNUSED_KSYMS handling to a dedicated target.
>>
>> The reason of commit 2441e78b1919 ("krbuild: better abstract vmlinux
>> sequential prerequisites") was to cater to CONFIG_BUILD_DOCSRC, but
>> it was later removed by commit 184892925118 ("samples: move blackfin
>> gptimers-example from Documentation").
>>
>> I also changed adjust_autoksyms.sh to simply exit with 1 or 0 to make
>> it look straightforward.
>
> That is wrong. If the script fails for some reason (it runs with -e set)
> then this will trigger an endless recursive make instead of failing the
> build.

Sorry, I missed this case.  You are right.

I will restore the original code.



-- 
Best Regards
Masahiro Yamada