Date:   Tue, 14 Jun 2022 15:30:14 +0300
From:   Jarkko Sakkinen <jarkko@...nel.org>
To:     Masami Hiramatsu <mhiramat@...nel.org>
Cc:     Song Liu <song@...nel.org>, Ard Biesheuvel <ardb@...nel.org>,
        Jarkko Sakkinen <jarkko@...fian.com>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Nathaniel McCallum <nathaniel@...fian.com>,
        Russell King <linux@...linux.org.uk>,
        Catalin Marinas <catalin.marinas@....com>,
        Will Deacon <will@...nel.org>,
        Thomas Bogendoerfer <tsbogend@...ha.franken.de>,
        "James E.J. Bottomley" <James.Bottomley@...senpartnership.com>,
        Helge Deller <deller@....de>,
        Michael Ellerman <mpe@...erman.id.au>,
        Benjamin Herrenschmidt <benh@...nel.crashing.org>,
        Paul Mackerras <paulus@...ba.org>,
        Paul Walmsley <paul.walmsley@...ive.com>,
        Palmer Dabbelt <palmer@...belt.com>,
        Albert Ou <aou@...s.berkeley.edu>,
        Heiko Carstens <hca@...ux.ibm.com>,
        Vasily Gorbik <gor@...ux.ibm.com>,
        Alexander Gordeev <agordeev@...ux.ibm.com>,
        Christian Borntraeger <borntraeger@...ux.ibm.com>,
        Sven Schnelle <svens@...ux.ibm.com>,
        "David S. Miller" <davem@...emloft.net>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        X86 ML <x86@...nel.org>, "H. Peter Anvin" <hpa@...or.com>,
        "Naveen N. Rao" <naveen.n.rao@...ux.ibm.com>,
        Anil S Keshavamurthy <anil.s.keshavamurthy@...el.com>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Kees Cook <keescook@...omium.org>,
        "Peter Zijlstra (Intel)" <peterz@...radead.org>,
        Nathan Chancellor <nathan@...nel.org>,
        Josh Poimboeuf <jpoimboe@...nel.org>,
        Mark Rutland <mark.rutland@....com>,
        "Eric W. Biederman" <ebiederm@...ssion.com>,
        Marco Elver <elver@...gle.com>,
        Dan Li <ashimida@...ux.alibaba.com>,
        Sami Tolvanen <samitolvanen@...gle.com>,
        "Russell King (Oracle)" <rmk+kernel@...linux.org.uk>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Linus Walleij <linus.walleij@...aro.org>,
        Chen Zhongjin <chenzhongjin@...wei.com>,
        Nicolas Pitre <nico@...xnic.net>,
        Mark Brown <broonie@...nel.org>,
        Luis Machado <luis.machado@...aro.org>,
        Geert Uytterhoeven <geert@...ux-m68k.org>,
        Joey Gouly <joey.gouly@....com>,
        Masahiro Yamada <masahiroy@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Andrey Konovalov <andreyknvl@...il.com>,
        Kefeng Wang <wangkefeng.wang@...wei.com>,
        Atsushi Nemoto <anemo@....ocn.ne.jp>,
        Guenter Roeck <linux@...ck-us.net>,
        Dave Anglin <dave.anglin@...l.net>,
        Christophe Leroy <christophe.leroy@...roup.eu>,
        Alexei Starovoitov <ast@...nel.org>,
        Nicholas Piggin <npiggin@...il.com>,
        Daniel Axtens <dja@...ens.net>,
        "Aneesh Kumar K.V" <aneesh.kumar@...ux.ibm.com>,
        Jordan Niethe <jniethe5@...il.com>,
        Guo Ren <guoren@...nel.org>, Anup Patel <anup@...infault.org>,
        Atish Patra <atishp@...shpatra.org>,
        Changbin Du <changbin.du@...el.com>,
        Heiko Stuebner <heiko@...ech.de>,
        Liao Chang <liaochang1@...wei.com>,
        Philipp Tomsich <philipp.tomsich@...ll.eu>,
        Wu Caize <zepan@...eed.com>,
        Emil Renner Berthing <kernel@...il.dk>,
        Alexander Egorenkov <egorenar@...ux.ibm.com>,
        Thomas Richter <tmricht@...ux.ibm.com>,
        Tobias Huschle <huschle@...ux.ibm.com>,
        Ilya Leoshkevich <iii@...ux.ibm.com>,
        Tom Lendacky <thomas.lendacky@....com>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        Michael Roth <michael.roth@....com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Javier Martinez Canillas <javierm@...hat.com>,
        Miroslav Benes <mbenes@...e.cz>,
        André Almeida <andrealmeid@...lia.com>,
        Tiezhu Yang <yangtiezhu@...ngson.cn>,
        Dmitry Torokhov <dmitry.torokhov@...il.com>,
        Aaron Tomlin <atomlin@...hat.com>,
        linux-arm-kernel@...ts.infradead.org, linux-mips@...r.kernel.org,
        linux-parisc@...r.kernel.org, linuxppc-dev@...ts.ozlabs.org,
        linux-riscv@...ts.infradead.org, linux-s390@...r.kernel.org,
        sparclinux@...r.kernel.org, linux-modules@...r.kernel.org
Subject: Re: [PATCH] kprobes: Enable tracing for mololithic kernel images

On Sun, Jun 12, 2022 at 09:30:41PM +0900, Masami Hiramatsu wrote:
> On Wed, 8 Jun 2022 11:19:19 -0700
> Song Liu <song@...nel.org> wrote:
> 
> > On Wed, Jun 8, 2022 at 9:28 AM Ard Biesheuvel <ardb@...nel.org> wrote:
> > >
> > > Hello Jarkko,
> > >
> > > On Wed, 8 Jun 2022 at 02:02, Jarkko Sakkinen <jarkko@...fian.com> wrote:
> > > >
> > > > Tracing with kprobes while running a monolithic kernel is currently
> > > > impossible because CONFIG_KPROBES is dependent of CONFIG_MODULES.  This
> > > > dependency is a result of kprobes code using the module allocator for the
> > > > trampoline code.
> > > >
> > > > Detaching kprobes from modules helps to squeeze down the user space,
> > > > e.g. when developing new core kernel features, while still having all
> > > > the nice tracing capabilities.
> > > >
> > > > For kernel/ and arch/*, move module_alloc() and module_memfree() to
> > > > module_alloc.c, and compile as part of vmlinux when either CONFIG_MODULES
> > > > or CONFIG_KPROBES is enabled.  In addition, flag kernel module specific
> > > > code with CONFIG_MODULES.
> > > >
> > > > As the result, kprobes can be used with a monolithic kernel.
> > >
> > > I think I may have mentioned this the previous time as well, but I
> > > don't think this is the right approach.
> > >
> > > Kprobes uses alloc_insn_page() to allocate executable memory, but the
> > > requirements for this memory are radically different compared to
> > > loadable modules, which need to be within an arch-specific distance of
> > > the core kernel, need KASAN backing etc etc.
> > 
> > I think the distance of core kernel requirement is the same for kprobe
> > alloc_insn_page and modules, no?
> 
> This strongly depends on how kprobes (software breakpoint and
> single-step) is implemented on the arch. For example, x86 implements
> the so-called "kprobe-booster" which jumps back from the single
> stepping trampoline buffer. Then the buffer address must be within
> the range where it can jump to the original address.
> However, if the arch implements single-step as an instruction
> emulation, it has no such limitation. As far as I know, arm64
> will do emulation for the instructions which change PC register
> and will do direct execution with another software breakpoint
> for other instructions.
> 
> Why I'm using module_alloc() for a generic function, is that
> can cover the limitation most widely.
> Thus, if we have CONFIG_ARCH_HAVE_ALLOC_INSN_PAGE flag and
> kprobes can check it instead of using __weak function, the
> kprobes may not need to depend on module_alloc() in general.

OK, I guess this is what Luis meant. 

I'll try to carve up something based on this.

BR, Jarkko

Powered by blists - more mailing lists