linux-kernel - Re: kprobes broken since 0d00449c7a28 ("x86: Replace ist_enter() with nmi

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20210128001353.66e7171b395473ef992d6991@kernel.org>
Date:   Thu, 28 Jan 2021 00:13:53 +0900
From:   Masami Hiramatsu <mhiramat@...nel.org>
To:     Nikolay Borisov <nborisov@...e.com>
Cc:     Peter Zijlstra <peterz@...radead.org>, mhiramat@...nel.org,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: kprobes broken since 0d00449c7a28
 ("x86: Replace ist_enter() with nmi_enter()")

Hi Nikolay,

On Wed, 27 Jan 2021 15:43:29 +0200
Nikolay Borisov <nborisov@...e.com> wrote:

> Hello,
> 
> I'm currently seeing latest Linus' master being somewhat broken w.r.t
> krpobes. In particular I have the following test-case:
> 
> #!/bin/bash
> 
> mkfs.btrfs -f /dev/vdc &> /dev/null
> mount /dev/vdc /media/scratch/
> 
> bpftrace -e 'kprobe:btrfs_sync_file {printf("kprobe: %s\n", kstack());}'
> &>bpf-output &
> bpf_trace_pid=$!
> 
> # force btrfs_sync_file to be called
> sleep 2
> xfs_io -f -c "pwrite 0 4m" -c "fsync" /media/scratch/file5
> 
> kill $bpf_trace_pid
> sleep 1
> 
> grep -q kprobe bpf-output
> retval=$?
> rm -f bpf-output
> umount /media/scratch
> 
> exit $retval
> 
> It traces btrfs_sync_file which is called when fsync is executed on a
> btrfs file, however I don't see the stacktrace being printed i.e the
> kprobe doesn't fire at all. The following alternative program:
> 
> bpftrace -e 'tracepoint:btrfs:btrfs_sync_file {printf("tracepoint:
> %s\n", kstack());} kprobe:btrfs_sync_file {printf("kprobe: %s\n",
> kstack());}'
> 
> only prints the stack from the tracepoint and not from the kprobe, given
> that the tracepoint is called from the btrfs_sync_file function.

Thank you for reporting!

If you don't mind, could you confirm it with ftrace (tracefs)?
bpftrace etc. involves too many things. It is better to test with
simpler way to test it.
I'm not familer with the bpftrace, but I think you can check it with

# cd /sys/kernel/debug/tracing
# echo p:myevent btrfs_sync_file >> kprobe_events
# echo stacktrace > events/kprobes/myevent/trigger
 (or echo 1 > options/stacktrace , if trigger file doesn't exist)

Could you also share your kernel config, so that we can reproduce it?

> 
> I started bisecting this and arrived at the following commit:
> 
> 0d00449c7a28 ("x86: Replace ist_enter() with nmi_enter()")
> 
> FWIW the following series is applied on the kernel I was testing:
> https://lore.kernel.org/lkml/159870598914.1229682.15230803449082078353.stgit@devnote2/
> 
> but it's still broken.

Peter, would you have any idea?

Thank you,

-- 
Masami Hiramatsu <mhiramat@...nel.org>