linux-kernel - Re: [RFC PATCH 0/3] kbuild: generate intermediate C files instead of copying

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Sat, 19 Aug 2017 19:31:06 +0800
From:   Cao jin <caoj.fnst@...fujitsu.com>
To:     Masahiro Yamada <yamada.masahiro@...ionext.com>,
        Sam Ravnborg <sam@...nborg.org>,
        Michal Marek <mmarek@...e.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        <linux-kbuild@...r.kernel.org>
CC:     <devicetree@...r.kernel.org>, Rob Herring <robh@...nel.org>,
        Jonathan Corbet <corbet@....net>,
        Richard Purdie <richard.purdie@...uxfoundation.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Nicholas Piggin <npiggin@...il.com>,
        <linux-doc@...r.kernel.org>,
        Markus Heiser <markus.heiser@...marit.de>,
        <linux-kernel@...r.kernel.org>,
        Frank Rowand <frowand.list@...il.com>,
        Rob Herring <robh+dt@...nel.org>,
        SeongJae Park <sj38.park@...il.com>,
        "Yann E. MORIN" <yann.morin.1998@...e.fr>
Subject: Re: [RFC PATCH 0/3] kbuild: generate intermediate C files instead of
 copying _shipped files

Hi,
  I am stuck in a similar problem recent days by chance.
  I am just curious about the purpose of introduction of these *_shipped
file, are they just for user's convenience when user doesn't install the
those tools?

-- 
Sincerely,
Cao jin

On 08/19/2017 04:49 PM, Masahiro Yamada wrote:
> In Linux build system convention, we do not run tools such as
> flex, bison, gperf during the kernel building.  Instead, manage
> generated C files in the repository with _shipped suffixes.
> They are simply shipped (copied) removing the _shipped suffixes
> during the kernel building.
> 
> Commit 7373f4f83c71 ("kbuild: add implicit rules for parser generation")
> added a mechanism to regenerate intermediate C files easily.
> The build rules are surrounded with ifdef REGENERATE_PARSERS.
> So, we need to pass REGENERATE_PARSERS=1 from the command line
> when we want to update them.
> 
> Here is one question.  Is it acceptable to use those rules all the time?
> That is, generate those C files by flex, bison, gperf during the
> kernel building.
> 
> This means, the build system depends on more external tools.
>>>From the users' point of view, they will need to install
> flex, bison, gperf in order to build the kernel.
>>>From the developers' point of view, the advantage is
> we do not need to version-control generated files, i.e. _shipped files
> will be deleted.
> 
> I'd like to know if this is acceptable or not.
> 
> For example, currently some files are simply shipped (copied)
> when building the kconfig program.
> 
>   $ make mrproper; make defconfig
>     HOSTCC  scripts/basic/fixdep
>     HOSTCC  scripts/kconfig/conf.o
>     SHIPPED scripts/kconfig/zconf.tab.c
>     SHIPPED scripts/kconfig/zconf.lex.c
>     SHIPPED scripts/kconfig/zconf.hash.c
>     HOSTCC  scripts/kconfig/zconf.tab.o
>     HOSTLD  scripts/kconfig/conf
>   *** Default configuration is based on 'x86_64_defconfig'
>   #
>   # configuration written to .config
>   #
> 
> With this series, they are created from *real* sources
> (*.y, *.l, *.gperf files).
> 
>   $ make mrproper; make defconfig
>     HOSTCC  scripts/basic/fixdep
>     HOSTCC  scripts/kconfig/conf.o
>     YACC    scripts/kconfig/zconf.tab.c
>     LEX     scripts/kconfig/zconf.lex.c
>     GPERF   scripts/kconfig/zconf.hash.c
>     HOSTCC  scripts/kconfig/zconf.tab.o
>     HOSTLD  scripts/kconfig/conf
>   *** Default configuration is based on 'x86_64_defconfig'
>   #
>   # configuration written to .config
>   #
> 
> Note:
> The tool versions in Documentation/process/changes.rst are just
> place-holders for now.  We need to figure out the minimal versions
> if we like to switch to this approach.
> 
> 
> 
> Masahiro Yamada (3):
>   kbuild: generate *.hash.c during build
>   kbuild: generate *.lex.c during build
>   kbuild: generate *.tab.c and *.tab.h during build
> 
>  Documentation/process/changes.rst        |   36 +
>  scripts/Makefile.lib                     |   26 +-
>  scripts/dtc/Makefile                     |    6 +-
>  scripts/dtc/dtc-lexer.lex.c_shipped      | 2259 ---------------------------
>  scripts/dtc/dtc-parser.tab.c_shipped     | 2303 ----------------------------
>  scripts/dtc/dtc-parser.tab.h_shipped     |  125 --
>  scripts/genksyms/Makefile                |    4 +-
>  scripts/genksyms/keywords.hash.c_shipped |  230 ---
>  scripts/genksyms/lex.lex.c_shipped       | 2291 ---------------------------
>  scripts/genksyms/parse.tab.c_shipped     | 2394 -----------------------------
>  scripts/genksyms/parse.tab.h_shipped     |  119 --
>  scripts/kconfig/Makefile                 |    1 +
>  scripts/kconfig/zconf.hash.c_shipped     |  297 ----
>  scripts/kconfig/zconf.lex.c_shipped      | 2473 ------------------------------
>  scripts/kconfig/zconf.tab.c_shipped      | 2471 -----------------------------
>  15 files changed, 53 insertions(+), 14982 deletions(-)
>  delete mode 100644 scripts/dtc/dtc-lexer.lex.c_shipped
>  delete mode 100644 scripts/dtc/dtc-parser.tab.c_shipped
>  delete mode 100644 scripts/dtc/dtc-parser.tab.h_shipped
>  delete mode 100644 scripts/genksyms/keywords.hash.c_shipped
>  delete mode 100644 scripts/genksyms/lex.lex.c_shipped
>  delete mode 100644 scripts/genksyms/parse.tab.c_shipped
>  delete mode 100644 scripts/genksyms/parse.tab.h_shipped
>  delete mode 100644 scripts/kconfig/zconf.hash.c_shipped
>  delete mode 100644 scripts/kconfig/zconf.lex.c_shipped
>  delete mode 100644 scripts/kconfig/zconf.tab.c_shipped
>