linux-kernel - Re: [PATCH] kbuild: use ARCH from compile.h in unclean source tree msg

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <e7b517ee-08a3-4bbe-a9c0-497f1469b04e@linuxfoundation.org>
Date: Wed, 7 May 2025 16:21:53 -0600
From: Shuah Khan <skhan@...uxfoundation.org>
To: Masahiro Yamada <masahiroy@...nel.org>
Cc: Nicolas Schier <nicolas.schier@...ux.dev>, nathan@...nel.org,
 brendan.higgins@...ux.dev, davidgow@...gle.com, rmoar@...gle.com,
 linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
 kunit-dev@...glegroups.com, linux-kselftest@...r.kernel.org,
 Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH] kbuild: use ARCH from compile.h in unclean source tree
 msg

On 5/7/25 01:23, Masahiro Yamada wrote:
> On Wed, May 7, 2025 at 7:07 AM Shuah Khan <skhan@...uxfoundation.org> wrote:
>>
>> On 5/6/25 05:12, Nicolas Schier wrote:
>>> On Fri, 02 May 2025, Shuah Khan wrote:
>>>
>>>> When make finds the source tree unclean, it prints a message to run
>>>> "make ARCH=x86_64 mrproper" message using the ARCH from the command
>>>> line. The ARCH specified in the command line could be different from
>>>> the ARCH of the existing build in the source tree.
>>>>
>>>> This could cause problems in regular kernel build and kunit workflows.
>>>>
>>>> Regular workflow:
>>>>
>>>> - Build x86_64 kernel
>>>>       $ make ARCH=x86_64
>>>> - Try building another arch kernel out of tree with O=
>>>>       $ make ARCH=um O=/linux/build
>>>> - kbuild detects source tree is unclean
>>>>
>>>>     ***
>>>>     *** The source tree is not clean, please run 'make ARCH=um mrproper'
>>>>     *** in /linux/linux_srcdir
>>>>     ***
>>>>
>>>> - Clean source tree as suggested by kbuild
>>>>       $ make ARCH=um mrproper
>>>> - Source clean appears to be clean, but it leaves behind generated header
>>>>     files under arch/x86
>>>>       arch/x86/realmode/rm/pasyms.h
>>>>
>>>> A subsequent x86_64e build fails with
>>>>     "undefined symbol sev_es_trampoline_start referenced ..."
>>>>
>>>> kunit workflow runs into this issue:
>>>>
>>>> - Build x86_64 kernel
>>>> - Run kunit tests:  it tries to build for user specified ARCH or uml
>>>>     as default:
>>>>       $ ./tools/testing/kunit/kunit.py run
>>>>
>>>> - kbuild detects unclean source tree
>>>>
>>>>     ***
>>>>     *** The source tree is not clean, please run 'make ARCH=um mrproper'
>>>>     *** in /linux/linux_6.15
>>>>     ***
>>>>
>>>> - Clean source tree as suggested by kbuild
>>>>       $ make ARCH=um mrproper
>>>> - Source clean appears to be clean, but it leaves behind generated header
>>>>     files under arch/x86
>>>>
>>>> The problem shows when user tries to run tests on ARCH=x86_64:
>>>>
>>>>       $ ./tools/testing/kunit/kunit.py run ARCH=x86_64
>>>>
>>>>       "undefined symbol sev_es_trampoline_start referenced ..."
>>>>
>>>> Build trips on arch/x86/realmode/rm/pasyms.h left behind by a prior
>>>> x86_64 build.
>>>>
>>>> Problems related to partially cleaned source tree are hard to debug.
>>>> Change Makefile to unclean source logic to use ARCH from compile.h
>>>> UTS_MACHINE string. With this change kbuild prints:
>>>>
>>>>       $ ./tools/testing/kunit/kunit.py run
>>>>
>>>>     ***
>>>>     *** The source tree is not clean, please run 'make ARCH=x86_64 mrproper'
>>>>     *** in /linux/linux_6.15
>>>>     ***
>>>>
>>>> Signed-off-by: Shuah Khan <skhan@...uxfoundation.org>
>>>> ---
>>>>    Makefile | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/Makefile b/Makefile
>>>> index 5aa9ee52a765..7ee29136b4da 100644
>>>> --- a/Makefile
>>>> +++ b/Makefile
>>>> @@ -674,7 +674,7 @@ ifeq ($(KBUILD_EXTMOD),)
>>>>                -d $(srctree)/include/config -o \
>>>>                -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \
>>
>> Would it make sense to check for include/generated as a catch all?
>>
>>>>               echo >&2 "***"; \
>>>> -            echo >&2 "*** The source tree is not clean, please run 'make$(if $(findstring command line, $(origin ARCH)), ARCH=$(ARCH)) mrproper'"; \
>>>> +            echo >&2 "*** The source tree is not clean, please run 'make ARCH=$(shell grep UTS_MACHINE $(srctree)/include/generated/compile.h | cut -d '"' -f 2) mrproper'"; \
>>>
>>> Please 'grep' option '-s'.
>>>
>>> There are some (rare) occassions, when there is no include/generated/compile.h
>>> but still the source tree will be considered to be dirty:
>>
>> I considered adding a check for not finding include/generated/compile.h
>> and figured if include/config is found we are probably safe.
>>
>> I will fix that.
> 
> 
> I do not think this patch makes sense.
> 
> Kbuild correctly detects that "the source tree is not clean enough
> to build with ARCH=um", and displays the following message:
>    ***
>    *** The source tree is not clean, please run 'make ARCH=um mrproper'
>    *** in /linux/linux_srcdir
>    ***
> 
> This is absolutely correct.

It detects it can't build um - but it doesn't detect that the
source tree is not clean. The problem is once user runs
'make ARCH=um mrproper' - these checks will find the source tree
clean even though it isn't - a subsequent x86_64 build could
fail.

If kbuild suggests mrproper for the existing kernel build instead,
the source tree will be cleaned correctly.

> 
> 
> The real issue is that "make ARCH=um mrproper"
> does not properly clean the source tree.
> 

The patch you sent doesn't solve the problem I am seeing.
You can find the details on the patch thread.

thanks,
-- Shuah