linux-kernel - Re: Build failures due to commit ea4054a2384 (handle huge number of modules)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130509041448.GA11398@roeck-us.net>
Date:	Wed, 8 May 2013 21:15:04 -0700
From:	Guenter Roeck <linux@...ck-us.net>
To:	Rusty Russell <rusty@...tcorp.com.au>
Cc:	linux-kernel@...r.kernel.org, linux-kbuild@...r.kernel.org,
	Fengguang Wu <fengguang.wu@...el.com>
Subject: Re: Build failures due to commit ea4054a2384 (handle huge number of
 modules)

Hi Rusty,

On Thu, May 09, 2013 at 10:26:00AM +0930, Rusty Russell wrote:
> Guenter Roeck <linux@...ck-us.net> writes:
> > Hi all,
> >
> > I started seeing build failures such as the following in the last week or so.
> >
> > make[2]: [__modpost] Error 1
> >
> > followed by lots of messages such as
> >
> > gcc: error: arch/x86/crypto/ablk_helper.mod.c: No such file or directory
> > gcc: fatal error: no input files
> >
> > (at least if I run make -i)
> 
> Hmm, weird!  Looks like __modules contains something it shouldn't.  We
> changed from:
> 
>         __modules := $(sort $(shell grep -h '\.ko$$' /dev/null $(wildcard $(MODVERDIR)/*.mod)))
> 
> To:
> 
>         MODLISTCMD := find $(MODVERDIR) -name '*.mod' | xargs -r grep -h '\.ko$$' | sort -u
>         __modules := $(shell $(MODLISTCMD))
> 
> They give the same results for me:
> 
> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> index 8dcdca2..1ca358d 100644
> --- a/scripts/Makefile.modpost
> +++ b/scripts/Makefile.modpost
> @@ -62,8 +62,14 @@ modulesymfile := $(firstword $(KBUILD_EXTMOD))/Module.symvers
>  # Step 1), find all modules listed in $(MODVERDIR)/
>  MODLISTCMD := find $(MODVERDIR) -name '*.mod' | xargs -r grep -h '\.ko$$' | sort -u
>  __modules := $(shell $(MODLISTCMD))
> +__old_modules := $(sort $(shell grep -h '\.ko$$' /dev/null $(wildcard $(MODVERDIR)/*.mod)))
>  modules   := $(patsubst %.o,%.ko, $(wildcard $(__modules:.ko=.o)))
>  
> +.PHONY: dump_modules
> +dump_modules:
> +	@echo $(__modules) > __modules.list
> +	@echo $(__old_modules) > __old_modules.list
> +
>  # Stop after building .o files if NOFINAL is set. Makes compile tests quicker
>  _modpost: $(if $(KBUILD_MODPOST_NOFINAL), $(modules:.ko:.o),$(modules))
>  
> @@ -84,7 +90,7 @@ quiet_cmd_modpost = MODPOST $(words $(filter-out vmlinux FORCE, $^)) modules
>        cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | $(modpost) -s -T -
>  
>  PHONY += __modpost
> -__modpost: $(modules:.ko=.o) FORCE
> +__modpost: $(modules:.ko=.o) dump_modules FORCE
>  	$(call cmd,modpost) $(wildcard vmlinux)
>  
>  quiet_cmd_kernel-mod = MODPOST $@
> 
I applied the above diff, and the output is exactly the same for me as well.
But the command still fails.

> > After reverting commit ea4054a238 (modpost: handle huge numbers of modules)
> > everything is fine.
> >
> > This happens with multiple gcc versions and target platforms (arm, x86_64,
> > i386, powerpc). Host kernel version is 3.8.10, system is Ubuntu 12.10.
> >
> > Does anyone else see this problem ? Any idea what might be wrong ?
> 
> Assuming it happens with a clean tree, please send the .config, and I'll
> see if I can reproduce...
> 
Attached is a failing configuration for x86_64. Note that
drivers/staging/android/logger.c fails to build with this configuration,
so you have to build it with make -i to see the problem.

Looking through my logs, it seems that the problem may be related to other
build errors. At least in my latest nightly builds there was always some other
build error when this happened.

Thanks,
Guenter

View attachment ".config" of type "text/plain" (49912 bytes)