linux-kernel - Re: [PATCH] kbuild: make clean rule robust against too long argument error

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230617200700.GA31221@lxhi-065>
Date:   Sat, 17 Jun 2023 22:07:00 +0200
From:   Eugeniu Rosca <erosca@...adit-jv.com>
To:     Masahiro Yamada <masahiroy@...nel.org>
CC:     <linux-kbuild@...r.kernel.org>, <linux-kernel@...r.kernel.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>,
        Eugeniu Rosca <erosca@...adit-jv.com>,
        Eugeniu Rosca <roscaeugeniu@...il.com>
Subject: Re: [PATCH] kbuild: make clean rule robust against too long argument
 error

Hello Yamada-san,

Many thanks for your feedback and for your patch.
It indeed addresses the issue I have reported in [1].

On Sun, Jun 18, 2023 at 12:30:25AM +0900, Masahiro Yamada wrote:
> Commit cd968b97c492 ("kbuild: make built-in.a rule robust against too
> long argument error") made a build rule robust against "Argument list
> too long" error.
> 
> Eugeniu Rosca reported the same error occurred when cleaning an external
> module.
> 
> The $(obj)/ prefix can be a very long path for external modules.

Confirmed. I am seeing an instance of $(obj) being 150 characters long,
due to an out-of-tree module deeply buried in a specific Yocto build.

In the current vanilla version of 'make clean' (w/o this patch), the
$(obj) prefix is applied to each and every file being removed,
dramatically increasing the strlen of arguments passed to 'rm -rf'.

> 
> Apply a similar solution to 'make clean'.
> 
> Reported-by: Eugeniu Rosca <erosca@...adit-jv.com>
> Signed-off-by: Masahiro Yamada <masahiroy@...nel.org>
> ---
> 
>  scripts/Makefile.clean | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
> index 3649900696dd..235408a44f90 100644
> --- a/scripts/Makefile.clean
> +++ b/scripts/Makefile.clean
> @@ -37,8 +37,9 @@ __clean-files   := $(wildcard $(addprefix $(obj)/, $(__clean-files)))
>  
>  # ==========================================================================
>  
> +# Use xargs to make this robust against "Argument list too long" error

Please, correct me if I am wrong, but it looks like the magic/brilliance
is in the 'patsubst' function, since below version also fails for me:

NOK: cmd_clean = printf '%s ' $(__clean-files) | xargs rm -rf

When running 'make clean' for my particular out-of-tree kernel module,
'patsubst' is able to decrease the total number of characters passed
to the shell's argument list from ~204k to ~47k, which elegantly
prevents the "Argument list too long" error.

>  quiet_cmd_clean = CLEAN   $(obj)
> -      cmd_clean = rm -rf $(__clean-files)
> +      cmd_clean = printf '$(obj)/%s ' $(patsubst $(obj)/%,%,$(__clean-files)) | xargs rm -rf
>  
>  __clean: $(subdir-ymn)
>  ifneq ($(strip $(__clean-files)),)
> -- 
> 2.39.2
> 

Since it fixes the problem reported in [1] (much appreciated):

Reviewed-by: Eugeniu Rosca <erosca@...adit-jv.com>
Tested-by: Eugeniu Rosca <erosca@...adit-jv.com>

[1] https://lore.kernel.org/linux-kbuild/20230616194505.GA27753@lxhi-065/

-- 
Best regards,
Eugeniu Rosca