linux-kernel - Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190312014522.jfxg3pdav2piuaqs@ast-mbp>
Date:   Mon, 11 Mar 2019 18:45:24 -0700
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     Steven Rostedt <rostedt@...dmis.org>
Cc:     Joel Fernandes <joel@...lfernandes.org>,
        Daniel Colascione <dancol@...gle.com>,
        Karim Yaghmour <karim.yaghmour@...rsys.com>,
        Geert Uytterhoeven <geert@...ux-m68k.org>,
        Greg KH <gregkh@...uxfoundation.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Alexei Starovoitov <ast@...nel.org>,
        atish patra <atishp04@...il.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Guenter Roeck <groeck@...omium.org>,
        Jonathan Corbet <corbet@....net>,
        Kees Cook <keescook@...omium.org>,
        Android Kernel Team <kernel-team@...roid.com>,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        linux-trace-devel@...r.kernel.org,
        Manoj Rao <linux@...ojrajarao.com>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Qais Yousef <qais.yousef@....com>,
        Randy Dunlap <rdunlap@...radead.org>,
        Shuah Khan <shuah@...nel.org>, Yonghong Song <yhs@...com>
Subject: Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to
 extend the kernel

On Mon, Mar 11, 2019 at 09:28:23PM -0400, Steven Rostedt wrote:
> On Mon, 11 Mar 2019 20:39:12 -0400
> Joel Fernandes <joel@...lfernandes.org> wrote:
> 
> > I think even though the kernel-headers can't have information about all data
> > structures, they do already contain a lot of data structure definitions we
> > need already. And anything needed can/should arguably be moved to include/ if
> > they are really needed for kernel extension by something "external" to the
> > kernel such as kernel modules or eBPF, right?
> 
> That's not my worry. I would like to be able to easily walk data
> structures from within the kernel, without having to do a lot of work
> in userspace to get that information. The kprobe_events could then be
> passed type casts or such to access data fields of arguments to
> functions and such.
> 
> > 
> > In any case, such a solution such as what Steve suggested, still cannot do
> > what we can with headers - such as build kernel modules on the fly using the
> > C-compiler without any auto-generation of C code from any debug artifiacts.
> > Think systemtap working with the module-backend without any need for
> > linux-headers package on the file system. So such a solution would still be a
> > bit orthogonal in scope to what this proposed solution can solve IMO.
> > 
> 
> With the information I would like to have, it would be trivial to read
> the data to create the header files needed for modules.

what you're asking for we already have. It's called BTF.
pahole takes vmlinux dwarf and convert it into ~1Mbyte of BTF that includes
all kernel types.
With gzip it can be compressed further if necessary.
We also have a prototype to generate all_vmlinux_types.h from BTF.
But it's not a substitute for kernel headers.
We've had a long discussion during last LPC regarding this:
http://vger.kernel.org/lpc-bpf2018.html#session-2
tldr: many tracing use cases will be solved with BTF, but kernel headers
are here to stay.