| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CALCETrWq4JEukFSnGLvHbgV7Vr1uYbMjYbcE_0RRBaC=Q6GO5Q@mail.gmail.com> Date: Fri, 10 Jul 2015 08:36:34 -0700 From: Andy Lutomirski <luto@...capital.net> To: Andrey Ryabinin <a.ryabinin@...sung.com> Cc: Sasha Levin <sasha.levin@...cle.com>, Ingo Molnar <mingo@...nel.org>, Denys Vlasenko <dvlasenk@...hat.com>, "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>, Brian Gerst <brgerst@...il.com>, Peter Zijlstra <peterz@...radead.org>, Borislav Petkov <bp@...en8.de>, "H. Peter Anvin" <hpa@...or.com>, Linus Torvalds <torvalds@...ux-foundation.org>, Oleg Nesterov <oleg@...hat.com>, Thomas Gleixner <tglx@...utronix.de>, Steven Rostedt <rostedt@...dmis.org> Subject: Re: [PATCH] x86/asm/entry/64: Clean up entry_64.S On Fri, Jul 10, 2015 at 8:26 AM, Andrey Ryabinin <a.ryabinin@...sung.com> wrote: > On 07/09/2015 03:59 AM, Andy Lutomirski wrote: >> Having failed to bisect, let's look at the trace: >> >> On Mon, Jul 6, 2015 at 8:00 AM, Sasha Levin <sasha.levin@...cle.com> wrote: >>> [3157054.661763] ------------[ cut here ]------------ >>> [3157054.662552] kernel BUG at arch/x86/kernel/nmi.c:533! >>> [3157054.663277] invalid opcode: 0000 [#1] PREEMPT SMP KASAN >>> [3157054.664164] Dumping ftrace buffer: >>> [3157054.664740] (ftrace buffer empty) >>> [3157054.665274] Modules linked in: >>> [3157054.665768] CPU: 16 PID: 11446 Comm: trinity-main Not tainted 4.1.0-next-20150703-sasha-00040-gd868f14-dirty #2292 >>> [3157054.667203] task: ffff880408813000 ti: ffff8803d29c8000 task.ti: ffff8803d29c8000 >>> [3157054.668256] RIP: do_nmi (arch/x86/kernel/nmi.c:533 (discriminator 1)) >>> [3157054.669378] RSP: 0018:ffff88077800bed8 EFLAGS: 00010006 >>> [3157054.670141] ================================================================== >>> [3157054.671268] BUG: KASan: out of bounds on stack in __show_regs+0x7f6/0x940 at addr ffff88077800be50 >> >> I bet that__show_regs interacts poorly with KASan for some reason. >> But that's not the underlying bug. In fact, the bad read is quite >> close the RSP, so this is almost certainly a bug in KASan or >> __show_regs. >> >>> [3157054.674604] Read of size 8 by task trinity-main/11446 >>> [3157054.676521] page:ffffea001de002c0 count:1 mapcount:0 mapping: (null) index:0x0 >>> [3157054.679451] flags: 0x42fffff80000400(reserved) >>> [3157054.681237] page dumped because: kasan: bad access detected >>> [3157054.683326] CPU: 16 PID: 11446 Comm: trinity-main Not tainted 4.1.0-next-20150703-sasha-00040-gd868f14-dirty #2292 >>> [3157054.687097] ffff88077800be50 000000009c65e33f ffff88077800b9f8 ffffffffa0ac8938 >>> [3157054.690303] 1ffffd4003bc0058 ffff88077800ba88 ffff88077800ba78 ffffffff9759796e >>> [3157054.693365] ffff88077800bab8 ffffffffa0abe0b3 0000000000000082 ffffffffa2fe39e4 >>> [3157054.696209] Call Trace: >>> [3157054.697180] <NMI> dump_stack (lib/dump_stack.c:52) >>> [3157054.699390] kasan_report_error (mm/kasan/report.c:132 mm/kasan/report.c:193) >>> [3157054.701663] ? printk (kernel/printk/printk.c:1896) >>> [3157054.703531] ? bitmap_weight (include/linux/bitmap.h:303) >>> [3157054.705553] __asan_report_load8_noabort (mm/kasan/report.c:230 mm/kasan/report.c:251) >>> [3157054.708014] ? __show_regs (arch/x86/kernel/process_64.c:68) >>> [3157054.710046] __show_regs (arch/x86/kernel/process_64.c:68) >>> [3157054.712066] ? printk (kernel/printk/printk.c:1896) >>> [3157054.713878] ? bitmap_weight (include/linux/bitmap.h:303) >>> [3157054.715875] ? start_thread_common.constprop.0 (arch/x86/kernel/process_64.c:58) >>> [3157054.718336] ? dump_stack_print_info (kernel/printk/printk.c:3121) >>> [3157054.720619] show_regs (arch/x86/kernel/dumpstack_64.c:313) >>> [3157054.722530] __die (arch/x86/kernel/dumpstack.c:294) >>> [3157054.724290] die (arch/x86/kernel/dumpstack.c:316) >>> [3157054.725962] do_trap (arch/x86/kernel/traps.c:214 arch/x86/kernel/traps.c:260) >>> [3157054.727805] do_error_trap (arch/x86/kernel/traps.c:298 include/linux/jump_label.h:125 include/linux/context_tracking_state.h:29 include/linux/context_tracking.h:46 arch/x86/kernel/traps.c:302) >>> [3157054.729843] ? do_device_not_available (arch/x86/kernel/traps.c:291) >>> [3157054.732211] ? do_nmi (arch/x86/kernel/nmi.c:533 (discriminator 1)) >>> [3157054.734101] ? kvm_clock_read (./arch/x86/include/asm/preempt.h:87 arch/x86/kernel/kvmclock.c:86) >>> [3157054.736165] ? sched_clock (arch/x86/kernel/tsc.c:305) >>> [3157054.738126] ? nmi_handle (arch/x86/kernel/nmi.c:134) >>> [3157054.740133] ? trace_hardirqs_off_thunk (arch/x86/entry/thunk_64.S:40) >>> [3157054.742997] do_invalid_op (arch/x86/kernel/traps.c:313) >>> [3157054.744991] invalid_op (arch/x86/entry/entry_64.S:925) >> >> So we got #UD somewhere... >> >>> [3157054.746873] ? do_nmi (arch/x86/kernel/nmi.c:533 (discriminator 1)) >>> [3157054.748769] ? do_nmi (arch/x86/kernel/nmi.c:515 arch/x86/kernel/nmi.c:531) >>> [3157054.750658] end_repeat_nmi (arch/x86/entry/entry_64.S:1435) >> >> ...here, perhaps? >> > > Huh? Perhaps here: > kernel BUG at arch/x86/kernel/nmi.c:533 > > We are in the middle of processing BUG_ON() > IOW, we hit BUG_ON(). BUG_ON() handler calls __show_regs(), for some reason kasan to complains here > and prints backtrace of bad access. Yeah, so something's wrong there. Cc: Steven. BTW, Steven, I think I thought up a very clean fix to the old RSP issue, and I'll email something out later today. I doubt it's the same thing here, though. > > So, after cutting off all kasan reports: > > [3157054.661763] ------------[ cut here ]------------ > [3157054.662552] kernel BUG at arch/x86/kernel/nmi.c:533! > [3157054.663277] invalid opcode: 0000 [#1] PREEMPT SMP KASAN > [3157054.664164] Dumping ftrace buffer: > [3157054.664740] (ftrace buffer empty) > [3157054.665274] Modules linked in: > [3157054.665768] CPU: 16 PID: 11446 Comm: trinity-main Not tainted 4.1.0-next-20150703-sasha-00040-gd868f14-dirty #2292 > [3157054.667203] task: ffff880408813000 ti: ffff8803d29c8000 task.ti: ffff8803d29c8000 > [3157054.668256] RIP: do_nmi (arch/x86/kernel/nmi.c:533 (discriminator 1)) > [3157054.669378] RSP: 0018:ffff88077800bed8 EFLAGS: 00010006 > [3157054.784428] RAX: 0000000080120001 RBX: 0000000000000001 RCX: 00000000c0000101 > [3157054.801838] RDX: 1ffffffff4691cd0 RSI: ffffffffa0c10620 RDI: ffffffffa344dc00 > [3157054.891910] RBP: ffff88077800bee8 R08: 0000000000000001 R09: 000000000000002e > [3157055.191450] pps pps0: PPS event at 4682.682479766 > [3157055.191456] pps pps0: capture assert seq #4932 > [3157055.196385] R10: ffffed014e1e4883 R11: ffffed014e1e4881 R12: ffff88077800bef8 > [3157055.416083] R13: 000b375311a5d4ab R14: ffffffffa3485190 R15: ffffffffa3485180 > [3157055.418637] FS: 00007f6d93c6f700(0000) GS:ffff880778000000(0000) knlGS:0000000000000000 > [3157055.421726] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [3157055.423798] CR2: 0000000004378000 CR3: 00000003d2987000 CR4: 00000000000007e0 > [3157055.426363] DR0: ffffffff81000000 DR1: 0000000000000000 DR2: 0000000000000000 > [3157055.428933] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600 > [3157055.431526] Stack: > [3157055.432310] 0000000000000001 0000000004378000 ffff88077800be98 ffffffffa0b2ff6f > [3157055.435066] ffffffffa3485180 ffffffffa3485190 000b375311a5d4ab 0000000000000000 > [3157055.437846] ffff88077800be98 dffffc0000000000 ffffed014e1e4881 ffffed014e1e4883 > [3157055.440612] Call Trace: > [3157055.441576] <NMI> > [3157055.442347] end_repeat_nmi (arch/x86/entry/entry_64.S:1435) > [3157055.444426] ? debug (arch/x86/entry/entry_64.S:1067) > [3157055.446211] ? debug (arch/x86/entry/entry_64.S:1067) > [3157055.447992] ? debug (arch/x86/entry/entry_64.S:1067) > [3157055.449762] <<EOE>> > [3157055.450579] <#DB> [3157055.451465] ? nmi_handle (arch/x86/kernel/nmi.c:134 include/linux/jump_label.h:125 include/trace/events/nmi.h:10 arch/x86/kernel/nmi.c:135) > [3157055.453456] <<EOE>> > >> Do you know what line 1435 was in the version you tested? There >> shouldn't be funny instructions in end_repeat_nmi, though. Did we end >> up off an instruction boundary? >> >> Here's my wild guess. The repeat_nmi thing is really rare. What if >> there's a CPU or emulator that can't do mov %cr2, %r12 or vice versa? >> mov from cr has a somewhat unusual encoding. What platform is this? >> Does KASan play games that would cause KVM to emulate a mov to or from >> cr2? >> > > I can't tell you what and how KVM emulates anything, but kasan just maps some memory at > certain location and read/writes it. That's it. My wild idea was clearly wrong, since that r12 thing happens on all NMIs. Also, I directly tested the KVM emulator, and it's fine. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists