linux-kernel - Re: External modules with O=... (was: Re: [PATCH] kbuild: Fix include path in scripts/Makefile.modpost)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAK7LNATDVN9ukAJdztobZ=aMLvfgE_wW1N_gsB-x3OiRE-Jrog@mail.gmail.com>
Date: Wed, 11 Sep 2024 16:28:30 +0900
From: Masahiro Yamada <masahiroy@...nel.org>
To: Lucas De Marchi <lucas.demarchi@...el.com>
Cc: Jing Leng <3090101217@....edu.cn>, Michal Marek <michal.lkml@...kovi.net>, 
	Nick Desaulniers <ndesaulniers@...gle.com>, 
	Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>, 
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>, Jing Leng <jleng@...arella.com>
Subject: Re: External modules with O=... (was: Re: [PATCH] kbuild: Fix include
 path in scripts/Makefile.modpost)

On Wed, Sep 11, 2024 at 1:36 PM Lucas De Marchi
<lucas.demarchi@...el.com> wrote:
>
> On Tue, Sep 10, 2024 at 09:43:14PM GMT, Lucas De Marchi wrote:
> >On Wed, Sep 11, 2024 at 09:10:09AM GMT, Jing Leng wrote:
> >>>-----Original Messages-----
> >>>From: "Lucas De Marchi" <lucas.demarchi@...el.com>
> >>>Send time:Tuesday, 09/10/2024 22:00:29
> >>>To: "Masahiro Yamada" <masahiroy@...nel.org>
> >>>Cc: 3090101217@....edu.cn, "Michal Marek" <michal.lkml@...kovi.net>, "Nick
> >>> Desaulniers" <ndesaulniers@...gle.com>, "Linux Kbuild mailing list" <linux-kbuild@...r.kernel.org>, "Linux Kernel Mailing List" <linux-kernel@...r.kernel.org>, "Jing Leng" <jleng@...arella.com>
> >>>Subject: External modules with O=... (was: Re: [PATCH] kbuild: Fix include path in scripts/Makefile.modpost)
> >>>
> >>>Hi, I was pointed to this thread since I'm trying something similar
> >>>in kmod's testsuite. See below.
> >>>
> >>>On Tue, May 24, 2022 at 02:52:45AM GMT, Masahiro Yamada wrote:
> >>>>On Tue, May 17, 2022 at 7:51 PM <3090101217@....edu.cn> wrote:
> >>>>>
> >>>>> From: Jing Leng <jleng@...arella.com>
> >>>>>
> >>>>> When building an external module, if users don't need to separate the
> >>>>> compilation output and source code, they run the following command:
> >>>>> "make -C $(LINUX_SRC_DIR) M=$(PWD)". At this point, "$(KBUILD_EXTMOD)"
> >>>>> and "$(src)" are the same.
> >>>>>
> >>>>> If they need to separate them, they run "make -C $(KERNEL_SRC_DIR)
> >>>>> O=$(KERNEL_OUT_DIR) M=$(OUT_DIR) src=$(PWD)". Before running the
> >>>>> command, they need to copy "Kbuild" or "Makefile" to "$(OUT_DIR)" to
> >>>>> prevent compilation failure.
> >>>>>
> >>>>> So the kernel should change the included path to avoid the copy operation.
> >>>>>
> >>>>> Signed-off-by: Jing Leng <jleng@...arella.com>
> >>>>> ---
> >>>>>  scripts/Makefile.modpost | 3 +--
> >>>>>  1 file changed, 1 insertion(+), 2 deletions(-)
> >>>>>
> >>>>> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> >>>>> index 48585c4d04ad..0273bf7375e2 100644
> >>>>> --- a/scripts/Makefile.modpost
> >>>>> +++ b/scripts/Makefile.modpost
> >>>>> @@ -87,8 +87,7 @@ obj := $(KBUILD_EXTMOD)
> >>>>>  src := $(obj)
> >>>>>
> >>>>>  # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS
> >>>>> -include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
> >>>>> -             $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
> >>>>> +include $(if $(wildcard $(src)/Kbuild), $(src)/Kbuild, $(src)/Makefile)
> >>>>>
> >>>>>  # modpost option for external modules
> >>>>>  MODPOST += -e
> >>>>> --
> >>>>> 2.17.1
> >>>>>
> >>>>
> >>>>
> >>>>I do not think "M=$(OUT_DIR) src=$(PWD)" is the official way,
> >>>>but this patch is a clean up.
> >>>
> >>>I tried what is in this patch and also tried to find an official way in
> >>>the docs.


There is no official way.

> >>>
> >>>In kmod's testsuite we build dummy kernel modules to exercise the API.
> >>>https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git/tree/testsuite/module-playground
> >>>
> >>>This works:
> >>>     make -C /lib/modules/$(uname -r)/build M=$PWD
> >>>
> >>>This doesn't:
> >>>     make -C /lib/modules/$(uname -r)/build M=$PWD O=/tmp/kmod_test_modules



O= points the output directory of the kernel,
not the output directory of the external modules.


/lib/modules/$(uname -r)/build
is the clean source tree.

/tmp/kmod_test_modules
contains the build artifacts of vmlinux and
in-tree modules.

Then, the command you gave would work.






> >>>
> >>>I also tried the variants above with setting src, but all of them give
> >>>me errors - I used 6.10 and 6.11-rc7 for these tests.
> >>>
> >>>Is there a way to do this?
> >>>
> >>>thanks
> >>>Lucas De Marchi
> >>>
> >>>>
> >>>>Applied to linux-kbuild. Thanks.
> >>>>
> >>>>
> >>>>--
> >>>>Best Regards
> >>>>Masahiro Yamada
> >>
> >>Hi Masahiro,
> >
> >I guess you meant Lucas :)
> >
> >>
> >>I think your intention is to separate the source code from the compiled output.
> >>The correct command should be:
> >>   make -C /lib/modules/$(uname -r)/build src=$PWD M=/tmp/kmod_test_modules
> >
> >oh, looks like this works. Apparently my mistake was trying to set O=
> >like I normally do for in-tree modules.
>
> spoke too early... It worked because I was in another machine pointing
> to a 6.8 kernel. It seems like something broke between 6.9 and 6.10.
>
> Running a quick bisect, it's pointing to this commit:
> 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")


Overriding 'src' from the command is not allowed. That's why.


> Error like below:
>
> $ make -j$(nproc) -C ~/p/linux-dim/src MddPWD/build srcx=$PWD
> make: Entering directory '/home/ldmartin/p/linux-dim/src'
> make[2]: *** No rule to make target '/home/ldmartin/p/kmod/testsuite/module-playground/build/mod-simple.o', needed by '/home/ldmartin/p/kmod/testsuite/module-playground/build/'.  Stop.
> make[1]: *** [/home/ldmartin/p/linux-dim/src/Makefile:1922: /home/ldmartin/p/kmod/testsuite/module-playground/build] Error 2
> make: *** [Makefile:240: __sub-make] Error 2
> make: Leaving directory '/home/ldmartin/p/linux-dim/src'


I suggested M=relative-path + VPATH=
but I do not know what you want to achieve.

https://lore.kernel.org/linux-kbuild/CAK7LNATGGibmjZzYX_A2SkJthmOPbKw2K3R7JYuHTWzgGL2Zjg@mail.gmail.com/







> Lucas De Marchi
>
> >
> >Thanks
> >Lucas De Marchi
> >
> >>
> >>You also can refer to:
> >>   https://github.com/lengjingzju/cbuild-ng/blob/main/scripts/core/inc.mod.mk
> >>1. The complete command is as follows:
> >>   make -C <Linux kernel source code directory> O=<Linux kernel compilation output directory> src=<Current driver module source code directory> M=<Current driver module compilation output directory>
> >>2. If the <Linux kernel source code directory> and the <Linux kernel compilation output directory> are the same, <O=xxx> can be omitted:
> >>   make -C <Linux kernel source code directory> src=<Current driver module source code directory> M=<Current driver module compilation output directory>
> >>2. If the <Current driver module source code directory> and the <Current driver module compilation output directory> are the same, <src=xxx> can be omitted:
> >>   make -C <Linux kernel source code directory> O=<Linux kernel compilation output directory> M=<Current driver module source code directory>
> >>
> >>Best Regards!
> >>Jing Leng



-- 
Best Regards
Masahiro Yamada