linux-kernel - Re: perf: fuzzer KASAN unwind_get_return

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20161117044828.vedc3whqkuki624r@treble>
Date:   Wed, 16 Nov 2016 22:48:28 -0600
From:   Josh Poimboeuf <jpoimboe@...hat.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Vince Weaver <vincent.weaver@...ne.edu>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        "davej@...emonkey.org.uk" <davej@...emonkey.org.uk>,
        "dvyukov@...gle.com" <dvyukov@...gle.com>,
        Stephane Eranian <eranian@...il.com>
Subject: Re: perf: fuzzer KASAN unwind_get_return_address

On Wed, Nov 16, 2016 at 03:58:49PM +0100, Peter Zijlstra wrote:
> 3BUG: KASAN: stack-out-of-bounds in unwind_get_return_address+0x1fb/0x220 at addr ffff88042f88bba0

So I dug through the disassembly (thanks for the vmlinux), and I'm
pretty sure the stack-out-of-bounds address is on the NMI stack, in the
kasan redzone in the stack frame of intel_pmu_handle_irq().

What's weird though is that perf_callchain_kernel() passes the pt_regs
from the IRQ, not from the NMI.  The unwinder should have started from
the IRQ stack.  But somehow it ended up unwinding to the middle of the
NMI stack.

So it seems like stack corruption in the IRQ or task stack, with a frame
pointer that points back to the middle of the NMI stack for some reason.
But then again, the kasan error report dumped the stack fine.  So that
would seem to rule out stack corruption...  So I have no idea what's
going on.

I got perf_fuzzer running and tried to recreate, but no luck.

Peter or Vince, can you try to recreate with this patch?  It dumps the
raw stack contents during a stack dump.  Hopefully that would give a
clue about what's going wrong.

diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c
index 499aa6f..67ff3ac 100644
--- a/arch/x86/kernel/dumpstack.c
+++ b/arch/x86/kernel/dumpstack.c
@@ -48,6 +48,30 @@ static void printk_stack_address(unsigned long address, int reliable,
 	printk("%s %s%pB\n", log_lvl, reliable ? "" : "? ", (void *)address);
 }

+static void raw_stack_dump(struct stack_info *info)
+{
+	unsigned long *s, word[4];
+	int skip = 0;
+
+	for (s = info->begin; s < info->end; s += 4) {
+		word[0] = READ_ONCE_NOCHECK(s[0]);
+		word[1] = READ_ONCE_NOCHECK(s[1]);
+		word[2] = READ_ONCE_NOCHECK(s[2]);
+		word[3] = READ_ONCE_NOCHECK(s[3]);
+
+		if (!word[0] && !word[1] && !word[2] && !word[3]) {
+			if (!skip)
+				printk("%p: %016x ...\n", s, 0);
+			skip = 1;
+			continue;
+		}
+
+		skip = 0;
+		printk("%p: %016lx %016lx %016lx %016lx\n",
+		       s, word[0], word[1], word[2], word[3]);
+	}
+}
+
 void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs,
 			unsigned long *stack, char *log_lvl)
 {
@@ -156,6 +180,8 @@ void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs,

 		if (str_end)
 			printk("%s <%s>\n", log_lvl, str_end);
+
+		raw_stack_dump(&stack_info);
 	}
 }