linux-kernel - Re: [PATCH v2] unicode: refactor the rule for regenerating utf8data.h

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20190501025930.GB5377@mit.edu>
Date:   Tue, 30 Apr 2019 22:59:30 -0400
From:   "Theodore Ts'o" <tytso@....edu>
To:     Masahiro Yamada <yamada.masahiro@...ionext.com>
Cc:     Olaf Weber <olaf@....com>,
        Gabriel Krisman Bertazi <krisman@...labora.co.uk>,
        Gabriel Krisman Bertazi <krisman@...labora.com>,
        linux-doc@...r.kernel.org, linux-kbuild@...r.kernel.org,
        linux-kernel@...r.kernel.org, Jonathan Corbet <corbet@....net>,
        Michal Marek <michal.lkml@...kovi.net>,
        linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH v2] unicode: refactor the rule for regenerating utf8data.h

On Mon, Apr 29, 2019 at 12:15:31PM +0900, Masahiro Yamada wrote:
> scripts/mkutf8data is used only when regenerating utf8data.h,
> which never happens in the normal kernel build. However, it is
> irrespectively built if CONFIG_UNICODE is enabled.
> 
> Moreover, there is no good reason for it to reside in the scripts/
> directory since it is only used in fs/unicode/.
> 
> Hence, move it from scripts/ to fs/unicode/.
> 
> In some cases, we bypass build artifacts in the normal build. The
> conventional way to do so is to surround the code with ifdef REGENERATE_*.
> 
> For example,
> 
>  - 7373f4f83c71 ("kbuild: add implicit rules for parser generation")
>  - 6aaf49b495b4 ("crypto: arm,arm64 - Fix random regeneration of S_shipped")
> 
> I rewrote the rule in a more kbuild'ish style.
> 
> In the normal build, utf8data.h is just shipped from the check-in file.
> 
> $ make
>   [ snip ]
>   SHIPPED fs/unicode/utf8data.h
>   CC      fs/unicode/utf8-norm.o
>   CC      fs/unicode/utf8-core.o
>   CC      fs/unicode/utf8-selftest.o
>   AR      fs/unicode/built-in.a
> 
> If you want to generate utf8data.h based on UCD, put *.txt files into
> fs/unicode/, then pass REGENERATE_UTF8DATA=1 from the command line.
> The mkutf8data tool will be automatically compiled to generate the
> utf8data.h from the *.txt files.
> 
> $ make REGENERATE_UTF8DATA=1
>   [ snip ]
>   HOSTCC  fs/unicode/mkutf8data
>   GEN     fs/unicode/utf8data.h
>   CC      fs/unicode/utf8-norm.o
>   CC      fs/unicode/utf8-core.o
>   CC      fs/unicode/utf8-selftest.o
>   AR      fs/unicode/built-in.a
> 
> I renamed the check-in utf8data.h to utf8data.h_shipped so that this
> will work for the out-of-tree build.
> 
> You can update it based on the latest UCD like this:
> 
> $ make REGENERATE_UTF8DATA=1 fs/unicode/
> $ cp fs/unicode/utf8data.h fs/unicode/utf8data.h_shipped
> 
> Also, I added entries to .gitignore and dontdiff.
> 
> Signed-off-by: Masahiro Yamada <yamada.masahiro@...ionext.com>
> ---
> 
> Changes in v2:
>  - Make this work correctly with O= option

Thanks, I've updated my tree to use this version of the commit.

	     	     	     	    	 - Ted