linux-kernel - Re: [PATCH 3/5] kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNAR4rJwrT2KLjLw-AbBvhO38xCZigC9C+DUVkn_5JM-KyQ@mail.gmail.com>
Date:   Wed, 2 Aug 2023 18:21:14 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     Eugeniu Rosca <erosca@...adit-jv.com>
Cc:     Nicolas Schier <n.schier@....de>,
        SzuWei Lin <szuweilin@...gle.com>,
        linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-arch@...r.kernel.org, Matthias.Thomae@...bosch.com,
        yyankovskyi@...adit-jv.com, Dirk.Behme@...bosch.com,
        Eugeniu Rosca <roscaeugeniu@...il.com>
Subject: Re: [PATCH 3/5] kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}

On Tue, Jul 25, 2023 at 6:24 PM Eugeniu Rosca <erosca@...adit-jv.com> wrote:
>
> Hello Yamada-san,
>
> Appreciate your willingness to support. Some findings below.
>
> On Sun, Jul 23, 2023 at 01:08:46AM +0900, Masahiro Yamada wrote:
> > On Thu, Jul 20, 2023 at 4:09 AM Eugeniu Rosca <erosca@...adit-jv.com> wrote:
> > > On Fri, Jun 23, 2023 at 04:45:44PM +0200, Eugeniu Rosca wrote:
>
> [..]
>
> > > > I will continue to increase my understanding behind what's happening.
> > > > In case there are already any suggestions, would appreciate those.
> > >
> > > JFYI, we've got confirmation from Qualcomm Customer Support interface
> > > that reverting [1] heals the issue on QC end as well. However, it looks
> > > like none of us has clear understanding how to properly
> > > troubleshoot/trace/compare the behavior before and after the commit.
> > >
> > > I would happily follow any suggestions.
> > >
> > > > [1] https://android.googlesource.com/kernel/common/+/bc6d3d83539512
> > > >     ("UPSTREAM: kbuild: rename cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}")
> > > >
> > > > [2] https://lore.kernel.org/linux-kbuild/20230616194505.GA27753@lxhi-065/
>
> [..]
>
> > Please backport 64d8aaa4ef388b22372de4dc9ce3b9b3e5f45b6c
> > and see if the problem goes away.
>
> Unfortunately, the problem remains after backporting the above commit.
>
> After some more bisecting and some more trial-and-error, I finally came
> up with a reproduction scenario against vanilla. It also shows that
> after reverting 7ce7e984ab2b21 ("kbuild: rename
> cmd_{bzip2,lzma,lzo,lz4,xzkern,zstd22}"), the problem goes away.
>
> It takes <30 seconds to reproduce the issue on my machine (on 2nd run).
>
> In order to make the test self-sufficient, it also clones the Linux
> sources (only during 1st run, with --depth 1, for minimal footprint),
> hence ~1.8 GB free space is required in /tmp .
>
> The repro.sh script:
>  => https://gist.github.com/erosca/1372fdc24126dc98031444613450c494
>
> Output against vanilla on 1st run (always OK, matches real-life case):
>  => https://gist.github.com/erosca/0f5b8e0a00a256d80f0c8a6364d81568
>
> Output against vanilla on 2nd/Nth run (NOK: Argument list too long):
>  => https://gist.github.com/erosca/e5c2c6479cc32244cc38d308deea4cf5
>
> Output against vanilla + revert_of_7ce7e984ab2b2 on Nth run (always OK):
>  => https://gist.github.com/erosca/57e114f92ea20132e19fc7f5a46e7c65
>
> Would it be possible to get your thoughts on the above?
>
> --
> Best regards,
> Eugeniu Rosca





Indeed, reverting 7ce7e984ab2b218d6e92d5165629022fe2daf9ee
makes qcom's external module build successfully
(but rebuilding is super slow).

Interestingly, revert 7ce7e984ab2b218d6e92d5165629022fe2daf9ee
then apply the attached patch, then
'Argument list too long' will come back.

So, this is unrelated to the actual build commands.




I suspect bare 'export', which expands all variables
while apparently most of them are not meant exported.



Insert the following in your reproducer, then it will work.


# qcom's audio-kernel sprinkles 'export' everywhere.
# Remove bare use of 'export'
find "$ABS_KMOD" -name Kbuild | xargs sed -i '/export$/d'




--
Best Regards
Masahiro Yamada

View attachment "0001-Add-dummy-commands-to-make-qcom-s-external-module-fa.patch" of type "text/x-patch" (1640 bytes)