linux-kernel - Re: Warnings/memory corruption in perf intel events

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 22 Jan 2016 07:36:28 -0500
From:	Sasha Levin <sasha.levin@...cle.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	Ingo Molnar <mingo@...nel.org>,
	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	"H. Peter Anvin" <hpa@...or.com>,
	LKML <linux-kernel@...r.kernel.org>,
	"x86@...nel.org" <x86@...nel.org>,
	Stephane Eranian <eranian@...gle.com>
Subject: Re: Warnings/memory corruption in perf intel events

On 08/10/2015 07:25 AM, Peter Zijlstra wrote:
> On Sun, Aug 09, 2015 at 03:46:48PM -0400, Sasha Levin wrote:
>> > Hi all,
>> > 
>> > While fuzzing with trinity inside a KVM tools guest running -next I've stumbled on the following:
> 
>> > [424256.911563] ==================================================================
>> > [424256.913989] BUG: KASan: use after free in intel_get_event_constraints+0xdb0/0xf90 at addr ffff8801741a70e9
>> > [424256.917044] Read of size 1 by task trinity-c162/20397
>> > [424256.918696] =============================================================================
>> > [424256.921309] BUG kmalloc-2048 (Tainted: G        W      ): kasan: bad access detected
> 
> Quite the puzzle that, and I'm not entirely sure I see how. The WARNs
> preceding this aren't giving me much confidence either, I've yet to find
> a way for them to happen.
> 
> That said, the only dynamically allocated memory here is managed on CPU
> hotplug. And we appear to consistently return NOTIFY_BAD if an
> allocation there fails, which should preclude the CPU hotplug from
> completing and therefore preclude events from forming on that CPU.
> 
> We do however appear to fail to NULL all our pointers, and even miss one
> kfree() in the error paths there (the constraint_list allocation fail,
> fails to free the shared_regs one).
> 
> Rework that code such that we kfree() all and always NULL our pointers
> after free.

I suspect that that patch didn't help, I'm seeing traces like this:

[ 1362.573349] WARNING: CPU: 3 PID: 13908 at arch/x86/kernel/cpu/perf_event.c:1345 x86_pmu_del+0x3e1/0x7f0()
[ 1362.574566] Modules linked in:
[ 1362.575017] CPU: 3 PID: 13908 Comm: syz-executor Not tainted 4.4.0-next-20160121-sasha-00020-g5e5e971-dirty #2811
[ 1362.576318]  1ffff10038db1e65 00000000a4877b67 ffff8801c6d8f3a8 ffffffff8344c8c1
[ 1362.577342]  0000000041b58ab3 ffffffff8f9763f5 ffffffff8344c7f6 ffff8800c33e0bc8
[ 1362.578371]  ffff8800c33e0bd0 00000000a4877b67 00000000a4877b67 0000000000000003
[ 1362.579217] Call Trace:
[ 1362.579536]  [<ffffffff8344c8c1>] dump_stack+0xcb/0x14a
[ 1362.580093]  [<ffffffff8344c7f6>] ? _atomic_dec_and_lock+0x106/0x106
[ 1362.580769]  [<ffffffff813bbda1>] warn_slowpath_common+0xe1/0x160
[ 1362.581410]  [<ffffffff811ce121>] ? x86_pmu_del+0x3e1/0x7f0
[ 1362.599438]  [<ffffffff813bc049>] warn_slowpath_null+0x29/0x30
[ 1362.600063]  [<ffffffff811ce121>] x86_pmu_del+0x3e1/0x7f0
[ 1362.600657]  [<ffffffff816bacfb>] event_sched_out+0x5ab/0x1480
[ 1362.601275]  [<ffffffff816ba750>] ? perf_pmu_enable+0x1c0/0x1c0
[ 1362.602016]  [<ffffffff816bc797>] ? __perf_remove_from_context+0x137/0x380
[ 1362.602854]  [<ffffffff816bc7a5>] __perf_remove_from_context+0x145/0x380
[ 1362.603675]  [<ffffffff816bc660>] ? __perf_event_disable+0x8a0/0x8a0
[ 1362.604463]  [<ffffffff816a30c0>] ? free_ctx+0x70/0x70
[ 1362.605104]  [<ffffffff816a320b>] remote_function+0x14b/0x200
[ 1362.605848]  [<ffffffff815ab368>] generic_exec_single+0x308/0x5d0
[ 1362.606632]  [<ffffffff815ab060>] ? cpumask_next+0xd0/0xd0
[ 1362.607359]  [<ffffffff834d0329>] ? check_preemption_disabled+0x39/0x270
[ 1362.608147]  [<ffffffff816bc660>] ? __perf_event_disable+0x8a0/0x8a0
[ 1362.608816]  [<ffffffff815ab992>] smp_call_function_single+0x122/0x4c0
[ 1362.624201]  [<ffffffff816a1086>] task_function_call+0x156/0x1a0
[ 1362.627453]  [<ffffffff816aa367>] event_function_call+0x1a7/0x310
[ 1362.630435]  [<ffffffff816aa89c>] perf_remove_from_context+0x10c/0x1b0
[ 1362.633337]  [<ffffffff816ca938>] put_event+0x318/0x510
[ 1362.637005]  [<ffffffff816cab8c>] perf_release+0x3c/0x60
[ 1362.649756]  [<ffffffff818e7ee5>] __fput+0x255/0x870
[ 1362.650347]  [<ffffffff818e8595>] ____fput+0x15/0x20
[ 1362.651060]  [<ffffffff8142ca66>] task_work_run+0x136/0x240
[ 1362.651716]  [<ffffffff813c7ffb>] do_exit+0x86b/0x1850
[ 1362.652369]  [<ffffffff813c7790>] ? mm_update_next_owner+0x6f0/0x6f0
[ 1362.653167]  [<ffffffff813e5edf>] ? __dequeue_signal+0x32f/0x730
[ 1362.653924]  [<ffffffff813c913c>] do_group_exit+0xec/0x390
[ 1362.654609]  [<ffffffff813f2f6a>] get_signal+0x5aa/0x1cb0
[ 1362.655276]  [<ffffffff8117aaed>] do_signal+0x8d/0x20d0
[ 1362.660705]  [<ffffffff8100557b>] exit_to_usermode_loop+0x1bb/0x270
[ 1362.661510]  [<ffffffff81006ded>] syscall_return_slowpath+0x3fd/0x590
[ 1362.662358]  [<ffffffff8d058aa2>] int_ret_from_sys_call+0x25/0x9f


Thanks,
Sasha