linux-kernel - Re: ftrace bug

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20220308102007.6c6618e7@gandalf.local.home>
Date:   Tue, 8 Mar 2022 10:20:07 -0500
From:   Steven Rostedt <rostedt@...dmis.org>
To:     Arend van Spriel <arend.vanspriel@...adcom.com>
Cc:     Ingo Molnar <mingo@...hat.com>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Russell King <linux@...linux.org.uk>,
        Alex Sverdlin <alexander.sverdlin@...ia.com>,
        Abel Vesa <abelvesa@...ux.com>, Rabin Vincent <rabin@....in>,
        Colin Cross <ccross@...roid.com>
Subject: Re: ftrace bug

[ Adding ARM folks (and those that touched the ftrace ARM port) as I don't
  know the ARM code ]

On Tue, 8 Mar 2022 11:53:35 +0100
Arend van Spriel <arend.vanspriel@...adcom.com> wrote:

> On 3/7/2022 2:24 PM, Steven Rostedt wrote:
> > On Mon, 7 Mar 2022 12:26:45 +0100
> > Arend van Spriel <arend.vanspriel@...adcom.com> wrote:
> >   
> >> Hi Steven,
> >>
> >> I wanted to use FTRACE on an ARM platform and I hit the following
> >> warning which results in ftrace bug. This happens upon loading a module.
> >> Looking up the warning I suspect the branch target is too far off. The
> >> module is quite large and therefor not loaded in the modules section. Is
> >> there a way to exclude a module. In ftrace_module_init I see a check for
> >> !mod->num_ftrace_callsites. Is there a way to avoid creating ftrace
> >> callsites in a module?  
> > 
> > Are you sure it's the size. Just want to make sure that's the reason before
> > going with different solutions.  
> 
> Interpreting the warning statement in insn.c and given the fact that the 
> module is loaded at 0xe3xxxxxx instead of kernel module space at 
> 0xbfxxxxxx I made this assumption, but ....
> 
> >>
> >> Regards,
> >> Arend
> >>
> >> ------------[ cut here ]------------
> >>
> >> WARNING: CPU: 2 PID: 1525 at arch/arm/kernel/insn.c:47
> >> __arm_gen_branch+0x70/0x78
> >>
> >> CPU: 2 PID: 1525 Comm: insmod Tainted: P                  4.19.183 #2
> >>
> >> Hardware name: Generic DT based system
> >>
> >> [<c02139e4>] (unwind_backtrace) from [<c020d4f8>] (show_stack+0x20/0x24)
> >>
> >> [<c020d4f8>] (show_stack) from [<c0922aec>] (dump_stack+0x98/0xac)
> >>
> >> [<c0922aec>] (dump_stack) from [<c0919b98>] (__warn.part.0+0xcc/0xe8)
> >>
> >> [<c0919b98>] (__warn.part.0) from [<c0919d4c>]
> >> (warn_slowpath_null+0x54/0x74)
> >> [<c0919d4c>] (warn_slowpath_null) from [<c021225c>]
> >> (__arm_gen_branch+0x70/0x78)
> >> [<c021225c>] (__arm_gen_branch) from [<c02120e4>]
> >> (ftrace_make_nop+0x64/0xec)
> >> [<c02120e4>] (ftrace_make_nop) from [<c02ac2b0>]
> >> (ftrace_process_locs+0x370/0x4b4)
> >>
> >> [<c02ac2b0>] (ftrace_process_locs) from [<c02af538>]
> >> (ftrace_module_init+0x38/0x3c)
> >>
> >> [<c02af538>] (ftrace_module_init) from [<c02a47d8>]
> >> (load_module+0x18d0/0x2570)
> >> [<c02a47d8>] (load_module) from [<c02a56f8>]
> >> (sys_finit_module+0xe0/0xf8)
> >> [<c02a56f8>] (sys_finit_module) from [<c0201000>]
> >> (ret_fast_syscall+0x0/0x58)
> >> Exception stack(0xd9b1bfa8 to 0xd9b1bff0)
> >>
> >> bfa0:                   01170228 00000000 00000003 01170228 00000000
> >> beb7ceb3
> >> bfc0: 01170228 00000000 beb7cdc4 0000017b 00127010 00000000 00000000
> >> 00000000
> >> bfe0: beb7cc38 beb7cc28 0001a0dc 00012890
> >>
> >> ---[ end trace f48808a851a4544a ]---
> >>
> >> ------------[ cut here ]------------
> >>
> >> WARNING: CPU: 2 PID: 1525 at kernel/trace/ftrace.c:2034
> >> ftrace_bug+0xfc/0x394  
> > 
> > Three should be more content after the "cut here" that is very relevant (I
> > hate that cut here, because I constantly need to tell people to show me
> > more :-p I need to add a "ftrace bug cut here" line.)
> > 
> > 
> > Could you show me the output right after that.  
> 
> ... here is the output I think you were looking for:
> 
> ftrace failed to modify 
> 
> [<a82ca82d>] (suspected corrupt symbol) 

Is the above "suspected corrupt symbol" kernel output? I don't see it in my
tree. What version of the kernel is this.

I'm guessing the above address a82ca82d is where it is trying to modify,
and is not in the symbol table. So something looks to be messed up.

> 
>   actual:   63:17:16:eb 

The above is what it found in that location.

Perhaps others have ideas.

-- Steve

> 
> Initializing ftrace call sites 
> 
> ftrace record flags: 2000000 
> 
>   (0) 
> 
>   expected tramp: c0211b88 
> 
> ftrace failed to modify 
> 
> [<6c88ec64>] (suspected corrupt symbol) 
> 
>   actual:   72:ab:08:eb 
> 
> Initializing ftrace call sites 
> 
> ftrace record flags: 2000000 
> 
>   (0) 
> 
>   expected tramp: c0211b88 
> 
> ftrace failed to modify 
> 
> [<b1ed303a>] (suspected corrupt symbol) 
> 
>   actual:   04:12:04:eb 
> 
> Initializing ftrace call sites 
> 
> ftrace record flags: 2000000 
> 
>   (0) 
> 
>   expected tramp: c0211b88 
> 
> ftrace failed to modify 
> 
> [<eb2fee66>] (suspected corrupt symbol) 
> 
>   actual:   d6:7e:00:eb 
> 
> Initializing ftrace call sites 
> 
> ftrace record flags: 2000000 
> 
>   (0) 
> 
>   expected tramp: c0211b88
> 
> Regards,
> Arend