linux-kernel - Re: [PATCH 21/27] kbuild, dma-mapping: benchmark: remove MODULE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <878rgopfdl.fsf@esperi.org.uk>
Date:   Thu, 23 Feb 2023 15:31:50 +0000
From:   Nick Alcock <nick.alcock@...cle.com>
To:     Luis Chamberlain <mcgrof@...nel.org>
Cc:     Christoph Hellwig <hch@....de>,
        Thomas Gleixner <tglx@...utronix.de>,
        Masahiro Yamada <masahiroy@...nel.org>,
        linux-modules@...r.kernel.org, linux-kernel@...r.kernel.org,
        Hitomi Hasegawa <hasegawa-hitomi@...itsu.com>,
        Marek Szyprowski <m.szyprowski@...sung.com>,
        iommu@...ts.linux.dev
Subject: Re: [PATCH 21/27] kbuild, dma-mapping: benchmark: remove
 MODULE_LICENSE in non-modules

On 22 Feb 2023, Luis Chamberlain spake thusly:

> On Wed, Feb 22, 2023 at 03:48:56PM +0100, Christoph Hellwig wrote:
>> Looks good:
>> 
>> Reviewed-by: Christoph Hellwig <hch@....de>
>> 
>> On Wed, Feb 22, 2023 at 12:14:47PM +0000, Nick Alcock wrote:
>> > Since commit 8b41fc4454e ("kbuild: create modules.builtin without
>> > Makefile.modbuiltin or tristate.conf"), MODULE_LICENSE declarations
>> > are used to identify modules.
>> 
>> .. but this seems like a really odd design.  How is this going to
>> continue working once we can autogenerate the module license section
>> from the SPDX tags, which we eventually really should?
>
> Yes I totally agree we should. But I think we should take this by steps.
> First, we ensure we have only MODULE_LICENSE() macros upstream on things which
> are really possible modules, ie we remove the false positives. We then put a
> stop-gap script which can complain if it finds new usecases which are buggy.

(and we have such a script already, though it's not in-tree: I used it
to generate the list of affected files that make up this series. I'll
keep running it at least once per release cycle to identify regressions
in this area, and fix them as they come up.)

> Then we look for an optimal way to address the final step:
>
>  * remove all MODULE_LICENSE() and autogenerate them from SPDX

Ooh that would be nice!

> The difficulty in this will be that we want to upkeep existing build
> heuristics and avoid to have to traverse the tree twice (see details
> on commit 8b41fc4454e). I can't think of an easy way to do this that
> does not involve using kconfig tristate somehow.

Nor can I -- and more generally I can't figure out a way to get from the
Kconfig symbols to the source files that constitute them without
retraversing the tree, since the only place the relationship is recorded
is in makefiles, and those makefiles use a lot of make functionality
(including more or less arbitrary make functions).

(restating the underlying difficulty here in case, like me, you lost
 track of it over the last few months)

Of course the build process is doing that traversal anyway -- the
problem is that the only approach we have to get from tristate to a list
of modules-or-builtins involves emitting *different values* for CONFIG_
symbols (uppercase rather than lowercase) and then triggering on those
to do things -- and if you do that you can't simultaneously use those
CONFIG_ variables for their normal purpose.

We can't rename those variables for this purpose because we're depending
on makefiles all across the tree expanding them. I tried to arrange for
their expansion to have side effects (so that evaluating $(CONFIG_FOO)
produced both 'y' or 'm' and *also* did... *something* that produced an
object file list for our consumption) but that also doesn't work because
unfortunately the tristate determination code needs a *mapping* from
CONFIG_ variable value to the result of the variable expansion, and a
line like

foo-$(CONFIG_FOO) := x y z

doesn't let the expansion of CONFIG_FOO have any sort of access to the
result of the assignment to foo-y / foo-m, and after the assignment's
happened there's no way to tell that the $(foo-m) -> x y z mapping was
generated by the expansion of CONFIG_FOO in particlar. So multiple
evaluations (which means, in effect, multiple make invocations) seems to
be the only way.

I do hope Masahiro has some brilliant idea here. Mind you I'm not sure
I'm clever enough to have come up with the original Makefile.modbuiltin
scheme either...