linux-kernel - Re: [PATCH v2 3/4] kbuild: support header-test-pattern-y

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <871rzfqnc5.fsf@intel.com>
Date:   Thu, 27 Jun 2019 14:28:26 +0300
From:   Jani Nikula <jani.nikula@...ux.intel.com>
To:     Masahiro Yamada <yamada.masahiro@...ionext.com>,
        linux-kbuild@...r.kernel.org
Cc:     Sam Ravnborg <sam@...nborg.org>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
        Jonathan Corbet <corbet@....net>,
        Michal Marek <michal.lkml@...kovi.net>
Subject: Re: [PATCH v2 3/4] kbuild: support header-test-pattern-y

On Thu, 27 Jun 2019, Masahiro Yamada <yamada.masahiro@...ionext.com> wrote:
> In my view, most of headers can be self-contained. So, it would be
> tedious to add every header to header-test-y explicitly. We usually
> end up with "all headers with some exceptions".
>
> There are two types in exceptions:
>
> [1] headers that are never compiled as standalone units
>
>   For examples, include/linux/compiler-gcc.h is not intended to be
>   included directly. We should always exclude such ones.
>
> [2] headers that are conditionally compiled as standalone units
>
>   Some headers can be compiled only for particular architectures.
>   For example, include/linux/arm-cci.h can be compiled only for
>   arm/arm64 because it requires <asm/arm-cci.h> to exist.
>   Clang can compile include/soc/nps/mtm.h only for arc because
>   it contains an arch-specific register in inline assembler.
>
> For [2], we can write Makefile like this:
>
>   header-test-$(CONFIG_ARM) += linux/arm-cci.h
>
> The new syntax header-test-pattern-y will be useful to specify
> "the rest".
>
> The typical usage is like this:
>
>   header-test-pattern-y += */*.h
>
> This adds all the headers in sub-directories to the test coverage,
> but headers added to header-test- are excluded. In this regards,
> header-test-pattern-y behaves like a weaker variant of header-test-y.
>
> Caveat:
> The patterns in header-test-pattern-y are prefixed with $(srctree)/$(src)/
> but not $(objtree)/$(obj)/. Stale generated patterns are often left over.
> For example, you will have ones when you traverse the git history for
> 'git bisect' without cleaning. If a wildcard is used for generated
> headers, it may match to stale headers.
>
> If you really want to compile-test generated headers, I recommend to
> add them to header-test-y explicitly. One pitfall is $(srctree)/$(src)/
> and $(objtree)/$(obj)/ point to the same directory for in-tree building.
> So, header-test-pattern-y should be used with care. It can potentially
> match to generated headers, which may be stale and fail to compile.
>
> Caveat2:
> You could use wildcard for header-test-. For example,
>
>   header-test- += asm-generic/%
>
> ... will exclude headers in asm-generic directory. Unfortunately, the
> wildcard character is '%' instead of '*' because this is evaluated by
> $(filter-out ...) whereas header-test-pattern-y is evaluated by
> $(wildcard ...). This is a kludge, but seems useful in some places...
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@...ionext.com>

Awesome! This will let us get rid of our local $(wildcard) hacks.

Tested-by: Jani Nikula <jani.nikula@...el.com>

> ---
>
> Changes in v2:
>   - New patch
>
>  Documentation/kbuild/makefiles.txt | 10 ++++++++++
>  scripts/Makefile.lib               | 10 ++++++++++
>  2 files changed, 20 insertions(+)
>
> diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt
> index 5080fec34609..b817e6cefb77 100644
> --- a/Documentation/kbuild/makefiles.txt
> +++ b/Documentation/kbuild/makefiles.txt
> @@ -1025,6 +1025,16 @@ When kbuild executes, the following steps are followed (roughly):
>  	i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled,
>  	this builds them as part of extra-y.
>  
> +    header-test-pattern-y
> +
> +	This works as a weaker version of header-test-y, and accepts wildcard
> +	patterns. The typical usage is:
> +
> +		  header-test-pattern-y += *.h
> +
> +	This specifies all the files that matches to '*.h' in the current
> +	directory, but the files in 'header-test-' are excluded.
> +
>  --- 6.7 Commands useful for building a boot image
>  
>  	Kbuild provides a few macros that are useful when building a
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 55ae1ec65342..54444933bbab 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -67,6 +67,16 @@ extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
>  endif
>  
>  # Test self-contained headers
> +
> +# Wildcard searches in $(srctree)/$(src)/, but not in $(objtree)/$(obj)/.
> +# Stale generated headers are often left over, so wildcard matching should
> +# be avoided. Please notice $(srctree)/$(src)/ and $(objtree)/$(obj) point
> +# to the same location for in-tree building.
> +header-test-y	+= $(filter-out $(header-test-), \
> +		$(patsubst $(srctree)/$(src)/%, %, \
> +		$(wildcard $(addprefix $(srctree)/$(src)/, \
> +		$(header-test-pattern-y)))))
> +
>  extra-$(CONFIG_HEADER_TEST) += $(addsuffix .s, $(header-test-y))
>  
>  # Add subdir path

-- 
Jani Nikula, Intel Open Source Graphics Center