linux-kernel - Re: [PATCH] kbuild: remove duplicate dependencies from .mod files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNARyYqmSByD3jQr+TPe1RHEj9CxqRE880AqxA5SU3yUk5A@mail.gmail.com>
Date:   Sun, 16 Feb 2020 13:27:20 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     Sami Tolvanen <samitolvanen@...gle.com>
Cc:     Michal Marek <michal.lkml@...kovi.net>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] kbuild: remove duplicate dependencies from .mod files

Hi Sami,

On Thu, Feb 13, 2020 at 2:13 AM Sami Tolvanen <samitolvanen@...gle.com> wrote:
>
> On Wed, Feb 12, 2020 at 5:23 AM Masahiro Yamada <masahiroy@...nel.org> wrote:
> > In which case are undefined symbols duplicated?
>
> When a module consists of multiple compilation units, which depend on
> the same external symbols. In Android, we ran into this when adding
> hardening features that all depend on an external error handler
> function with a rather long name. When CONFIG_TRIM_UNUSED_SYMS was
> later enabled, we ran into this:
>
> $ llvm-nm drivers/gpu/drm/nouveau/nouveau.o | sed -n 's/^  *U //p' |
> xargs echo | wc
>       2    9136  168660
>
> xargs defaults to 128kiB limit for command line size, so the output
> was split into two lines, which means some of the dependencies were
> dropped and we ran into modpost errors. One method of fixing this is
> to increase the limit:
>
> $ llvm-nm drivers/gpu/drm/nouveau/nouveau.o | sed -n 's/^  *U //p' |
> xargs -s 262144 echo | wc
>       1    9136  168660
>
> But it seems removing duplicates is a better solution as the length of
> the dependency list is reduced significantly:
>
> $ llvm-nm drivers/gpu/drm/nouveau/nouveau.o | sed -n 's/^  *U //p' |
> sort -u | xargs echo | wc
>       1    2716   50461


At least, I am unable to reproduce this in upstream.

This is my result for x86 allmodconfig builds.

masahiro@...ver:~/workspace/linux-kbuild$ nm
drivers/gpu/drm/nouveau/nouveau.o | sed -n 's/^  *U //p' | wc
    572     572   11478
masahiro@...ver:~/workspace/linux-kbuild$ nm
drivers/gpu/drm/nouveau/nouveau.o | sed -n 's/^  *U //p' | sort -u |
wc
    572     572   11478


I see no difference with/without 'sort -u'.


I also tried llvm-nm instead of GNU nm,
but the result is the same.


masahiro@...ver:~/workspace/linux-kbuild$ llvm-nm
drivers/gpu/drm/nouveau/nouveau.o | sed -n 's/^  *U //p' | wc
    572     572   11478
masahiro@...ver:~/workspace/linux-kbuild$ llvm-nm
drivers/gpu/drm/nouveau/nouveau.o | sed -n 's/^  *U //p' | sort -u |
wc
    572     572   11478




> > Do you have a .config to reproduce it?
>
> I can currently reproduce this on an Android kernel that has
> Control-Flow Integrity (CFI) enabled. While this feature is not
> upstreamed yet, there's nothing that would prevent us from hitting the
> command line limit with sufficiently large modules otherwise as well.


Does ACK do this differently?

I think it would be strange
if $(NM) duplicated undefined symbols.



-- 
Best Regards
Masahiro Yamada