linux-kernel - Re: [PATCH v3 4/4] kbuild: compile-test kernel headers to ensure they are self-contained

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190628180057.GA22758@ravnborg.org>
Date:   Fri, 28 Jun 2019 20:00:57 +0200
From:   Sam Ravnborg <sam@...nborg.org>
To:     Masahiro Yamada <yamada.masahiro@...ionext.com>
Cc:     linux-kbuild@...r.kernel.org, Jani Nikula <jani.nikula@...el.com>,
        Tony Luck <tony.luck@...el.com>,
        John Fastabend <john.fastabend@...il.com>,
        Jakub Kicinski <jakub.kicinski@...ronome.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        xdp-newbies@...r.kernel.org, Anton Vorontsov <anton@...msg.org>,
        Matthias Brugger <matthias.bgg@...il.com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        Michal Marek <michal.lkml@...kovi.net>,
        Jesper Dangaard Brouer <hawk@...nel.org>,
        Martin KaFai Lau <kafai@...com>,
        linux-mediatek@...ts.infradead.org,
        linux-arm-kernel@...ts.infradead.org,
        Colin Cross <ccross@...roid.com>,
        "David S. Miller" <davem@...emloft.net>,
        Kees Cook <keescook@...omium.org>,
        Alexei Starovoitov <ast@...nel.org>, netdev@...r.kernel.org,
        linux-kernel@...r.kernel.org, bpf@...r.kernel.org
Subject: Re: [PATCH v3 4/4] kbuild: compile-test kernel headers to ensure
 they are self-contained

Hi Masahiro.

On Fri, Jun 28, 2019 at 01:39:02AM +0900, Masahiro Yamada wrote:
> The headers in include/ are globally used in the kernel source tree
> to provide common APIs. They are included from external modules, too.
> 
> It will be useful to make as many headers self-contained as possible
> so that we do not have to rely on a specific include order.
> 
> There are more than 4000 headers in include/. In my rough analysis,
> 70% of them are already self-contained. With efforts, most of them
> can be self-contained.
> 
> For now, we must exclude more than 1000 headers just because they
> cannot be compiled as standalone units. I added them to header-test-.
> The black list was mostly generated by a script, so should be checked
> later.
The list is smaller than I had expected.
And I see why you insisted on avoiding a maze ok Kbuild files.
It looks good, except there is a few issues..


The file kernel/kheaders_data.tar.xz includes all the .s files.
Something needs to be done to exclude the .s files...

When building a full kernel the build fails like this:
  LD      vmlinux.o
aarch64-linux-gnu-ld: cannot find include/lib.a: No such file or directory
make[1]: *** [/home/sam/kernel/linux-kbuild.git/Makefile:1054: vmlinux] Error 1
make[1]: Leaving directory '/home/sam/kernel/linux-kbuild.git/.build/arm64-allyesconfig'
make: *** [Makefile:179: sub-make] Error 2


include/uapi/linux/mman.h fails when building sparc64 allmodconfig.
There is likely more header files that will fail when we start to
throw this after diverse randconfigs.
I have no good idea how to catch this.
Unless your scripts could automate this across several architectures.

I did not continue my testing futher.

> +header-test-			+= uapi/drm/vmwgfx_drm.h
> +header-test-			+= uapi/linux/a.out.h
> +header-test-			+= uapi/linux/coda.h
...
> +header-test-			+= uapi/xen/evtchn.h
> +header-test-			+= uapi/xen/gntdev.h
> +header-test-			+= uapi/xen/privcmd.h

I though uapi files were covered by another Makefile?
If they are added because we pull them in using a pattern, maybe they
should be removed using a specific filer-out?

	Sam