linux-kernel - Re: [regression ?] kbuild: fix building bzImage with CONFIG_TRIM_UNUSED

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LFD.2.20.1612101506520.1657@knanqh.ubzr>
Date:   Sat, 10 Dec 2016 16:04:09 -0500 (EST)
From:   Nicolas Pitre <nicolas.pitre@...aro.org>
To:     Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>
cc:     Jarod Wilson <jarod@...hat.com>, Michal Marek <mmarek@...e.com>,
        linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        Sergey Senozhatsky <sergey.senozhatsky@...il.com>
Subject: Re: [regression ?] kbuild: fix building bzImage with CONFIG_TRIM_UNUSED_KSYMS
 enabled

On Sat, 10 Dec 2016, Sergey Senozhatsky wrote:

> On (12/09/16 13:07), Nicolas Pitre wrote:
> [..]
> > > build:
> > > make -j4 > build_log 2>&1
> > > 
> > > package:
> > > make -j4 INSTALL_MOD_PATH="${pkgdir}" modules_install >> build_log 2>&1
> > 
> > Weird.
> 
> it is. sorry for long reply, it took me some time to track it down.
> turned out, the script also does `prepare' and `kernelrelease'. so
> the sequence of commands in my build script is
> 
> 	make prepare
> 	make kernelrelease
> # functon build
> 	make -j4
> # finction package
> 	make -j4 INSTALL_MOD_PATH=XXXX modules_install
> 
> 
> now. the problem here is that, apparently, and I didn't know that,
> "make prepare" and "make kernelrelease" are executed twice.
> 
> - first time when I build the kernel
>  make prepare
>  make kernelrelease
>  make -j4
> 
> - second time when I install the modules
>  make prepare
>  make kernelrelease
>  make -j4 INSTALL_MOD_PATH=XXXX modules_install
> 
> 
> so this will not install modules:
>  make prepare; make kernelrelease; make -j4; make prepare; make kernelrelease; make -j4 INSTALL_MOD_PATH=/tmp/MODULES modules_install
> 
> and this will:
>  make prepare; make kernelrelease; make -j4; make kernelrelease; make -j4 INSTALL_MOD_PATH=/tmp/MODULES modules_install

Right. And this is because of this in the main Makefile:

# Create temporary dir for module support files
# clean it up only when building all modules
cmd_crmodverdir = $(Q)mkdir -p $(MODVERDIR) \
                  $(if $(KBUILD_MODULES),; rm -f $(MODVERDIR)/*)

The list of modules to install is created from files found in 
$(MODVERDIR)/. This is cleared when KBUILD_MODULES is set.  Oddly 
enough, KBUILD_MODULES is _not_ globally set when building individual 
modules probably not to clear MODVERDIR. This requires explicit override 
like in this rule:

%.ko: prepare scripts FORCE
	$(cmd_crmodverdir)
	$(Q)$(MAKE) KBUILD_MODULES=$(if $(CONFIG_MODULES),1)  \
	  $(build)=$(build-dir) $(@:.ko=.o)

One could wonder why $(cmd_crmodverdir) is executed again here given 
that it is already part of the "prepare" target, but that's an 
orthogonal issue.

Another question is whether or not KBUILD_MODULES is the right criteria 
for clearing MODVERDIR. My first reaction is to say it is not, but I 
can't come up with anything better at the moment.

And KBUILD_MODULES must be set for any target that results in 
vmlinux being built (and there are many of them including arch specific) 
whenever CONFIG_TRIM_UNUSED_KSYMS=y. Can this be enforced elsewhere in 
the Makefile, like in the recipe for $(vmlinux-dirs)? I don't know. IMHO 
this will only make things even less pretty than they are now.

In the mean time, though, I'm wondering why you have to do "make 
prepare" twice, or even at all.  Semantically, we could think of 
"prepare" as meaning to set things up for the build.  That could imply 
the erasing of some temporary files or even product files. Therefore 
that shouldn't be appropriate before a "modules_install" IMHO. 
Furthermore the "prepare" target is not listed amongst the documented 
make targets neither in the README file nor in the "make help" output.

So, given all the above considerations, would it be possible for you to 
"fix" your build script instead?

Nicolas