linux-kernel - Re: 5.15 ftrace bug when enabling function

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20220805182552.742c4162@gandalf.local.home>
Date:   Fri, 5 Aug 2022 18:25:52 -0400
From:   Steven Rostedt <rostedt@...dmis.org>
To:     Ivan Babrou <ivan@...udflare.com>
Cc:     Daniel Dao <dqminh@...udflare.com>,
        kernel-team <kernel-team@...udflare.com>,
        Ingo Molnar <mingo@...hat.com>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        Masami Hiramatsu <mhiramat@...nel.org>
Subject: Re: 5.15 ftrace bug when enabling function_graph

On Fri, 5 Aug 2022 13:14:47 -0700
Ivan Babrou <ivan@...udflare.com> wrote:

> On Fri, Aug 5, 2022 at 11:53 AM Steven Rostedt <rostedt@...dmis.org> wrote:
> >
> > On Fri, 5 Aug 2022 11:37:56 -0700
> > Ivan Babrou <ivan@...udflare.com> wrote:
> >  
> > > root@foo:~# echo 'bpf_dispatcher_*_func' >
> > > /sys/kernel/tracing/set_ftrace_notrace
> > >
> > > root@foo:~# trace-cmd record -p function_graph -l handle_mm_fault -P 3367417
> > > plugin 'function_graph'
> > > Hit Ctrl^C to stop recording  
> >
> >  
> > > [1459709.204253] ------------[ ftrace bug ]------------
> > > [1459709.204257] ftrace failed to modify
> > > [1459709.204262] [<ffffffffac3ea960>] bpf_dispatcher_xdp_func+0x0/0x10
> > > [1459709.204275]  actual:   ffffffe9:ffffff9b:76:ffffffb0:14
> > > [1459709.204302] Setting ftrace call site to call ftrace function
> > > [1459709.204305] ftrace record flags: 10000001
> > > [1459709.204309]  (1)
> > >                   expected tramp: ffffffffabc4bcb0
> > > [1459709.204325] ------------[ cut here ]------------  
> >
> > Did you reboot since the error happened? Because the command you showed
> > should not have touched bpf_dispatcher_xdp_func.  
> 
> I did reboot, but I can reproduce if you have things for me to try.

Let's see if this works manually.

Reboot again.

Make sure the above "ftrace bug" is not there.

If not already mounted, as root:

 # mount -t tracefs tracefs /sys/krenel/tracing
 # cd /sys/kernel/tracing
 # cat enabled_functions

Let me know what is there.

 # echo handle_mm_fault > set_ftrace_filter
 # cat set_ftrace_filter

to make sure that handle_mm_fault is listed.

 # echo function_graph > current_tracer

If it fails again, let me know. It should not, but if it does, it needs to
be investigated.

Oh, and what exactly is this kernel version. The backtrace you posted shows:

  5.15.19-cloudflare-2022.2.1

If you can not reproduce this on a vanilla 5.15.19 kernel, then it may be a
bad backport of some patch.

-- Steve