linux-kernel - Re: [PATCH v3 4/4] kbuild: compile-test kernel headers to ensure they are self-contained

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAK7LNAQDqtm5F_JoPAjPOuf6s3d0F1=Ctyq6s0u2DWNpbFr5vg@mail.gmail.com>
Date:   Sun, 30 Jun 2019 16:57:23 +0900
From:   Masahiro Yamada <yamada.masahiro@...ionext.com>
To:     Sam Ravnborg <sam@...nborg.org>
Cc:     Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Jani Nikula <jani.nikula@...el.com>,
        Tony Luck <tony.luck@...el.com>,
        John Fastabend <john.fastabend@...il.com>,
        Jakub Kicinski <jakub.kicinski@...ronome.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        xdp-newbies@...r.kernel.org, Anton Vorontsov <anton@...msg.org>,
        Matthias Brugger <matthias.bgg@...il.com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        Michal Marek <michal.lkml@...kovi.net>,
        Jesper Dangaard Brouer <hawk@...nel.org>,
        Martin KaFai Lau <kafai@...com>,
        "moderated list:ARM/Mediatek SoC support" 
        <linux-mediatek@...ts.infradead.org>,
        linux-arm-kernel <linux-arm-kernel@...ts.infradead.org>,
        Colin Cross <ccross@...roid.com>,
        "David S. Miller" <davem@...emloft.net>,
        Kees Cook <keescook@...omium.org>,
        Alexei Starovoitov <ast@...nel.org>,
        Networking <netdev@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        bpf@...r.kernel.org
Subject: Re: [PATCH v3 4/4] kbuild: compile-test kernel headers to ensure they
 are self-contained

Hi Sam,


On Sat, Jun 29, 2019 at 3:01 AM Sam Ravnborg <sam@...nborg.org> wrote:
>
> Hi Masahiro.
>
> On Fri, Jun 28, 2019 at 01:39:02AM +0900, Masahiro Yamada wrote:
> > The headers in include/ are globally used in the kernel source tree
> > to provide common APIs. They are included from external modules, too.
> >
> > It will be useful to make as many headers self-contained as possible
> > so that we do not have to rely on a specific include order.
> >
> > There are more than 4000 headers in include/. In my rough analysis,
> > 70% of them are already self-contained. With efforts, most of them
> > can be self-contained.
> >
> > For now, we must exclude more than 1000 headers just because they
> > cannot be compiled as standalone units. I added them to header-test-.
> > The black list was mostly generated by a script, so should be checked
> > later.
> The list is smaller than I had expected.
> And I see why you insisted on avoiding a maze ok Kbuild files.
> It looks good, except there is a few issues..
>
>
> The file kernel/kheaders_data.tar.xz includes all the .s files.
> Something needs to be done to exclude the .s files...

Good catch. I will change scripts/gen_kheaders.sh


> When building a full kernel the build fails like this:
>   LD      vmlinux.o
> aarch64-linux-gnu-ld: cannot find include/lib.a: No such file or directory
> make[1]: *** [/home/sam/kernel/linux-kbuild.git/Makefile:1054: vmlinux] Error 1
> make[1]: Leaving directory '/home/sam/kernel/linux-kbuild.git/.build/arm64-allyesconfig'
> make: *** [Makefile:179: sub-make] Error 2

My bad - I built only include/,
without testing full build.

I will fix.


>
> include/uapi/linux/mman.h fails when building sparc64 allmodconfig.
> There is likely more header files that will fail when we start to
> throw this after diverse randconfigs.
> I have no good idea how to catch this.
> Unless your scripts could automate this across several architectures.

Thanks. I excluded a little more headers.


> I did not continue my testing futher.
>
> > +header-test-                 += uapi/drm/vmwgfx_drm.h
> > +header-test-                 += uapi/linux/a.out.h
> > +header-test-                 += uapi/linux/coda.h
> ...
> > +header-test-                 += uapi/xen/evtchn.h
> > +header-test-                 += uapi/xen/gntdev.h
> > +header-test-                 += uapi/xen/privcmd.h
>
> I though uapi files were covered by another Makefile?
> If they are added because we pull them in using a pattern, maybe they
> should be removed using a specific filer-out?

I have not looked at this closely yet.

usr/include/Makefile tests UAPI headers
crafted by scripts/headers_install.sh

Testing UAPI headers in their raw form
makes sense, I think.



-- 
Best Regards
Masahiro Yamada