linux-kernel - Re: [PATCH] kbuild: use ARCH from compile.h in unclean source tree msg

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <9458d0d0-3096-4b77-bd20-3c051b082b57@linuxfoundation.org>
Date: Wed, 7 May 2025 16:48:29 -0600
From: Shuah Khan <skhan@...uxfoundation.org>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: Nicolas Schier <nicolas.schier@...ux.dev>, nathan@...nel.org,
 brendan.higgins@...ux.dev, davidgow@...gle.com, rmoar@...gle.com,
 linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
 kunit-dev@...glegroups.com, linux-kselftest@...r.kernel.org,
 Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH] kbuild: use ARCH from compile.h in unclean source tree
 msg

On 5/7/25 16:21, Shuah Khan wrote:
> On 5/7/25 01:23, Masahiro Yamada wrote:
>> On Wed, May 7, 2025 at 7:07 AM Shuah Khan <skhan@...uxfoundation.org> wrote:
>>>
>>> On 5/6/25 05:12, Nicolas Schier wrote:
>>>> On Fri, 02 May 2025, Shuah Khan wrote:
>>>>
>>>>> When make finds the source tree unclean, it prints a message to run
>>>>> "make ARCH=x86_64 mrproper" message using the ARCH from the command
>>>>> line. The ARCH specified in the command line could be different from
>>>>> the ARCH of the existing build in the source tree.
>>>>>
>>>>> This could cause problems in regular kernel build and kunit workflows.
>>>>>
>>>>> Regular workflow:
>>>>>
>>>>> - Build x86_64 kernel
>>>>>       $ make ARCH=x86_64
>>>>> - Try building another arch kernel out of tree with O=
>>>>>       $ make ARCH=um O=/linux/build
>>>>> - kbuild detects source tree is unclean
>>>>>
>>>>>     ***
>>>>>     *** The source tree is not clean, please run 'make ARCH=um mrproper'
>>>>>     *** in /linux/linux_srcdir
>>>>>     ***
>>>>>
>>>>> - Clean source tree as suggested by kbuild
>>>>>       $ make ARCH=um mrproper
>>>>> - Source clean appears to be clean, but it leaves behind generated header
>>>>>     files under arch/x86
>>>>>       arch/x86/realmode/rm/pasyms.h
>>>>>
>>>>> A subsequent x86_64e build fails with
>>>>>     "undefined symbol sev_es_trampoline_start referenced ..."
>>>>>
>>>>> kunit workflow runs into this issue:
>>>>>
>>>>> - Build x86_64 kernel
>>>>> - Run kunit tests:  it tries to build for user specified ARCH or uml
>>>>>     as default:
>>>>>       $ ./tools/testing/kunit/kunit.py run
>>>>>
>>>>> - kbuild detects unclean source tree
>>>>>
>>>>>     ***
>>>>>     *** The source tree is not clean, please run 'make ARCH=um mrproper'
>>>>>     *** in /linux/linux_6.15
>>>>>     ***
>>>>>
>>>>> - Clean source tree as suggested by kbuild
>>>>>       $ make ARCH=um mrproper
>>>>> - Source clean appears to be clean, but it leaves behind generated header
>>>>>     files under arch/x86
>>>>>
>>>>> The problem shows when user tries to run tests on ARCH=x86_64:
>>>>>
>>>>>       $ ./tools/testing/kunit/kunit.py run ARCH=x86_64
>>>>>
>>>>>       "undefined symbol sev_es_trampoline_start referenced ..."
>>>>>
>>>>> Build trips on arch/x86/realmode/rm/pasyms.h left behind by a prior
>>>>> x86_64 build.
>>>>>
>>>>> Problems related to partially cleaned source tree are hard to debug.
>>>>> Change Makefile to unclean source logic to use ARCH from compile.h
>>>>> UTS_MACHINE string. With this change kbuild prints:
>>>>>
>>>>>       $ ./tools/testing/kunit/kunit.py run
>>>>>
>>>>>     ***
>>>>>     *** The source tree is not clean, please run 'make ARCH=x86_64 mrproper'
>>>>>     *** in /linux/linux_6.15
>>>>>     ***
>>>>>
>>>>> Signed-off-by: Shuah Khan <skhan@...uxfoundation.org>
>>>>> ---
>>>>>    Makefile | 2 +-
>>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/Makefile b/Makefile
>>>>> index 5aa9ee52a765..7ee29136b4da 100644
>>>>> --- a/Makefile
>>>>> +++ b/Makefile
>>>>> @@ -674,7 +674,7 @@ ifeq ($(KBUILD_EXTMOD),)
>>>>>                -d $(srctree)/include/config -o \
>>>>>                -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \
>>>
>>> Would it make sense to check for include/generated as a catch all?
>>>
>>>>>               echo >&2 "***"; \
>>>>> -            echo >&2 "*** The source tree is not clean, please run 'make$(if $(findstring command line, $(origin ARCH)), ARCH=$(ARCH)) mrproper'"; \
>>>>> +            echo >&2 "*** The source tree is not clean, please run 'make ARCH=$(shell grep UTS_MACHINE $(srctree)/include/generated/compile.h | cut -d '"' -f 2) mrproper'"; \
>>>>
>>>> Please 'grep' option '-s'.
>>>>
>>>> There are some (rare) occassions, when there is no include/generated/compile.h
>>>> but still the source tree will be considered to be dirty:
>>>
>>> I considered adding a check for not finding include/generated/compile.h
>>> and figured if include/config is found we are probably safe.
>>>
>>> I will fix that.
>>
>>
>> I do not think this patch makes sense.
>>
>> Kbuild correctly detects that "the source tree is not clean enough
>> to build with ARCH=um", and displays the following message:
>>    ***
>>    *** The source tree is not clean, please run 'make ARCH=um mrproper'
>>    *** in /linux/linux_srcdir
>>    ***
>>
>> This is absolutely correct.
> 
> It detects it can't build um - but it doesn't detect that the
> source tree is not clean. The problem is once user runs
> 'make ARCH=um mrproper' - these checks will find the source tree
> clean even though it isn't - a subsequent x86_64 build could
> fail.

kbuild can no longer detect that the tree is unclean since
mrproper um deletes files and directories kbuild checks to
determine if the source tree is unclean.

These failures are hard to debug and wastes lot of time.

thanks,
-- Shuah