linux-kernel - Re: [PATCH] kbuild: Run syncconfig with -s

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAK7LNAT4nRBkc1Cg5tJCCo=pmfLpXiAehv8tdh-KUK66JAPbxQ@mail.gmail.com>
Date:   Mon, 14 Sep 2020 22:32:25 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     Ingo Molnar <mingo@...nel.org>
Cc:     Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Michal Marek <michal.lkml@...kovi.net>
Subject: Re: [PATCH] kbuild: Run syncconfig with -s

On Mon, Sep 14, 2020 at 9:37 PM Ingo Molnar <mingo@...nel.org> wrote:
>
>
> * Masahiro Yamada <masahiroy@...nel.org> wrote:
>
> > On Thu, Aug 20, 2020 at 3:35 PM Ingo Molnar <mingo@...nel.org> wrote:
> > >
> > > On every kernel build that runs --syncconfig, there's an output of the following line:
> > >
> > >   scripts/kconfig/conf  --syncconfig Kconfig
> > >
> > > This is the only non-platform build message the kbuild system emits that isn't
> > > prefixed by at least a space, or is a build warning.
> > >
> > > Run it under -s - if there's any problem it will emit messages anyway.
> > >
> > > With this change the following simple grep filter will show all build warnings
> > > and errors of a kernel build:
> > >
> > >    make | grep -v '^ '
> >
> >
> >
> > I do want to see something when syncconfig is invoked.
> >
> > I will apply this instead:
> > https://patchwork.kernel.org/patch/11727445/
>
> BTW., there's another, rather spurious bug I recently triggered in kbuild.
>
> Occasionally when I Ctrl-C a kernel build on a system with a lot of CPUs,
> the .o.cmd file gets corrupted:
>
>   mm/.pgtable-generic.o.cmd:5: *** unterminated call to function 'wildcard': missing ')'.  Stop.
>   make: *** [Makefile:1788: mm] Error 2
>   make: *** Waiting for unfinished jobs....
>
> The .o.cmd file is half-finished:
>
>     $(wildcard include/config/shmem.h) \
>     $(wildcard include/config/hugetlb/page.h) \
>     $(wildcard include/config/zone/device.h) \
>     $(wildcard include/config/dev/pagemap/ops.h) \
>     $(wildcard include/config/device/private.h) \
>     $(wildcard include/config/pci/p2pdma.h) \
>     $(wildcard include/config/sparsemem.h) \
>     $(wildcard include/config/sparsemem/vmemmap.h) \
>     $(wildcard include/config/numa/balancing.h) \
>     $(wildcard i
>     [premature EOF]


Hmm, previously fixdep was not robust against
errors or interruption.
So, I occasionally observed the same problem as you saw.


I fixed it in
6f9ac9f4427ec0470ccffbf852cfaf326677cc21


Since then, I have not seen
the broken .cmd file issue.


I repeated 'make -j256' and Ctrl-C,
but I could not reproduce it...



If GNU Make was interrupted during the build rule
of mm/pgtable-generic.o, GNU Make should automatically
have deleted mm/pgtable-generic.o because
partially updated (i.e. corrupted) files are
the source of all the troubles.

Isn't this the case for you?


If I interrupt Kbuild, I see several
Deleting file '.....'
messages, like follows:



  CC      fs/seq_file.o
  CC      drivers/acpi/dock.o
  CC      drivers/scsi/scsi_devinfo.o
  CC      fs/xattr.o
^Cmake[1]: *** [scripts/Makefile.build:283: fs/seq_file.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: net/ipv6/af_inet6.o] Interrupt
make[3]: *** [scripts/Makefile.build:283:
drivers/net/phy/mdio-boardinfo.o] Interrupt
make[4]: *** [scripts/Makefile.build:283:
drivers/tty/serial/8250/8250_pnp.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: security/selinux/status.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: arch/x86/kernel/dumpstack.o] Interrupt
make[2]: *** Deleting file 'drivers/acpi/resource.o'
make[2]: *** [scripts/Makefile.build:500: drivers/net/phy] Interrupt
make[2]: *** [scripts/Makefile.build:283: net/sched/act_api.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: drivers/scsi/scsi_devinfo.o] Interrupt
make[1]: *** Deleting file 'arch/x86/pci/mmconfig_64.o'
make[1]: *** [scripts/Makefile.build:500: net/ipv6] Interrupt
make[2]: *** [scripts/Makefile.build:283: net/ethtool/wol.o] Interrupt
make[1]: *** Deleting file 'arch/x86/pci/init.o'
make[1]: *** [scripts/Makefile.build:283: arch/x86/pci/mmconfig_64.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: net/sunrpc/socklib.o] Interrupt
make[1]: *** [scripts/Makefile.build:283: fs/xattr.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: sound/hda/hdac_sysfs.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: net/mac80211/main.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: drivers/input/input-mt.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: net/rfkill/core.o] Interrupt
make[1]: *** [scripts/Makefile.build:283: lib/bug.o] Interrupt
make: *** [Makefile:1792: lib] Interrupt
make[2]: *** Deleting file 'drivers/rtc/systohc.o'




> Instead of the regular rules that end in:
>
>     $(wildcard include/config/memory/hotplug/sparse.h) \
>
>     mm/pgtable-generic.o: $(deps_mm/pgtable-generic.o)
>
>     $(deps_mm/pgtable-generic.o):
>     [regular EOF]
>
> Manually removing the corrupted .o.cmd dot file solves the bug.
>
> There's no reproducer other than Ctrl-C-ing large build jobs a couple of times.
>
> Thanks,
>
>         Ingo


--
Best Regards
Masahiro Yamada