linux-kernel - Re: [RFC][PATCH 00/12] various perf fixes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CACT4Y+Zdn2i0bG3=ALRkm2_RwAJE9WTf9qYb=5vXfDtos8Hzgg@mail.gmail.com>
Date:	Tue, 12 Jan 2016 12:01:22 +0100
From:	Dmitry Vyukov <dvyukov@...gle.com>
To:	Ingo Molnar <mingo@...nel.org>
Cc:	Peter Zijlstra <peterz@...radead.org>,
	alexander.shishkin@...ux.intel.com,
	Stephane Eranian <eranian@...gle.com>,
	LKML <linux-kernel@...r.kernel.org>, vince@...ter.net,
	Andi Kleen <andi@...stfloor.org>, jolsa@...hat.com,
	Thomas Gleixner <tglx@...utronix.de>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Arnaldo Carvalho de Melo <acme@...radead.org>
Subject: Re: [RFC][PATCH 00/12] various perf fixes

On Tue, Jan 12, 2016 at 12:00 PM, Dmitry Vyukov <dvyukov@...gle.com> wrote:
> On Tue, Jan 12, 2016 at 11:57 AM, Dmitry Vyukov <dvyukov@...gle.com> wrote:
>> On Tue, Jan 12, 2016 at 11:11 AM, Ingo Molnar <mingo@...nel.org> wrote:
>>>
>>> * Peter Zijlstra <peterz@...radead.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've been hunting perf bugs for the past few weeks. This resulted in this pile
>>>> of patches, that mostly seems to work -- I still get an occasional fail so
>>>> something is still off.
>>>>
>>>> But I've been sitting on this for far too long, so here goes.
>>>>
>>>> (I've not at all tested perf-cgroup)
>>>>
>>>> The code compiles in between patches, but I've not bothered trying to run/test
>>>> any intermediate stage. There's just too many inter-related fail.
>>>>
>>>> Alexander, Stephane, could you guys please have a hard look at this?
>>>>
>>>> Andi, Dmitry, patch 10 should explain the getting stuck in
>>>> perf_install_in_context() forever thing you both have observed.
>>>
>>> Btw., if there's no test failures I plan to apply and push this to Linus fairly
>>> soon, so guys please give it all the review and testing you can.
>>
>>
>> Yes, I was able to apply it on top of the latest Linus tree. Thanks.
>> Now testing perf with these patches at full capacity.
>
> Hit this in a minute:
>
> ------------[ cut here ]------------
> WARNING: CPU: 0 PID: 12905 at kernel/events/core.c:2651
> task_ctx_sched_out+0x8d/0xa0()
> Modules linked in:
> CPU: 0 PID: 12905 Comm: syz-executor Not tainted 4.4.0+ #224
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  00000000ffffffff ffff88003290f4e8 ffffffff8290d92d 0000000000000000
>  ffff8800348c5f00 ffffffff85fc05c0 ffff88003290f528 ffffffff813429e9
>  ffffffff81617ffd ffffffff85fc05c0 0000000000000a5b ffff880035c09188
> Call Trace:
>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>  [<ffffffff8290d92d>] dump_stack+0x6f/0xa2 lib/dump_stack.c:50
>  [<ffffffff813429e9>] warn_slowpath_common+0xd9/0x140 kernel/panic.c:483
>  [<ffffffff81342c19>] warn_slowpath_null+0x29/0x30 kernel/panic.c:516
>  [<ffffffff81617ffd>] task_ctx_sched_out+0x8d/0xa0 kernel/events/core.c:2651
>  [<     inline     >] perf_event_context_sched_out kernel/events/core.c:2550
>  [<ffffffff81622263>] __perf_event_task_sched_out+0x5d3/0x1230
> kernel/events/core.c:2634
>  [<     inline     >] perf_event_task_sched_out include/linux/perf_event.h:948
>  [<     inline     >] prepare_task_switch kernel/sched/core.c:2613
>  [<     inline     >] context_switch kernel/sched/core.c:2771
>  [<ffffffff85e7d2d7>] __schedule+0xb27/0x1c50 kernel/sched/core.c:3282
>  [<ffffffff85e7eed2>] preempt_schedule_common+0x42/0x70 kernel/sched/core.c:3352
>  [<ffffffff85e7ef17>] _cond_resched+0x17/0x20 kernel/sched/core.c:4720
>  [<     inline     >] __wait_for_common kernel/sched/completion.c:90
>  [<     inline     >] wait_for_common kernel/sched/completion.c:101
>  [<ffffffff85e8158b>] wait_for_completion+0x8b/0x300
> kernel/sched/completion.c:122
>  [<ffffffff8149255e>] __wait_rcu_gp+0x12e/0x1a0 kernel/rcu/update.c:365
>  [<ffffffff81498218>] synchronize_sched.part.58+0x88/0xb0 kernel/rcu/tree.c:3220
>  [<ffffffff814a19a3>] synchronize_sched+0xa3/0x120 kernel/rcu/tree.c:3217
>  [<     inline     >] synchronize_rcu include/linux/rcupdate.h:320
>  [<ffffffff817dd6be>] namespace_unlock+0xee/0x100 fs/namespace.c:1349
>  [<ffffffff817e671e>] drop_collected_mounts+0x8e/0xa0 fs/namespace.c:1762
>  [<ffffffff817eaf1c>] put_mnt_ns+0x4c/0x70 fs/namespace.c:3149
>  [<ffffffff813a7594>] free_nsproxy+0x44/0x1d0 kernel/nsproxy.c:161
>  [<ffffffff813a7992>] switch_task_namespaces+0xa2/0xc0 kernel/nsproxy.c:213
>  [<ffffffff813a79c7>] exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:218
>  [<ffffffff8134bb69>] do_exit+0x8b9/0x2b80 kernel/exit.c:749
>  [<ffffffff8134dfa8>] do_group_exit+0x108/0x330 kernel/exit.c:880
>  [<ffffffff81371214>] get_signal+0x5e4/0x1500 kernel/signal.c:2307
>  [<ffffffff81193db3>] do_signal+0x83/0x1c90 arch/x86/kernel/signal.c:712
>  [<ffffffff81006685>] exit_to_usermode_loop+0x1a5/0x210
> arch/x86/entry/common.c:247
>  [<     inline     >] prepare_exit_to_usermode arch/x86/entry/common.c:282
>  [<ffffffff8100851a>] syscall_return_slowpath+0x2ba/0x340
> arch/x86/entry/common.c:344
>  [<ffffffff85e8dba2>] int_ret_from_sys_call+0x25/0x9f
> arch/x86/entry/entry_64.S:281
> ---[ end trace 93985d046a082b72 ]---


And this:

------------[ cut here ]------------
WARNING: CPU: 3 PID: 24027 at kernel/events/core.c:2744
ctx_sched_in+0x253/0x1390()
Modules linked in:
CPU: 3 PID: 24027 Comm: syz-executor Not tainted 4.4.0+ #224
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
 00000000ffffffff ffff8800620079d0 ffffffff8290d92d 0000000000000000
 ffff880065652f80 ffffffff85fc05c0 ffff880062007a10 ffffffff813429e9
 ffffffff8161bd13 ffffffff85fc05c0 0000000000000ab8 0000000000000003
Call Trace:
 [<     inline     >] __dump_stack lib/dump_stack.c:15
 [<ffffffff8290d92d>] dump_stack+0x6f/0xa2 lib/dump_stack.c:50
 [<ffffffff813429e9>] warn_slowpath_common+0xd9/0x140 kernel/panic.c:483
 [<ffffffff81342c19>] warn_slowpath_null+0x29/0x30 kernel/panic.c:516
 [<ffffffff8161bd13>] ctx_sched_in+0x253/0x1390 kernel/events/core.c:2744
 [<ffffffff8161ce97>] perf_event_sched_in+0x47/0xa0 kernel/events/core.c:2106
 [<ffffffff8161cfcc>] ctx_resched+0xdc/0x1d0 kernel/events/core.c:2119
 [<ffffffff8161db6b>] __perf_install_in_context+0x12b/0x1b0
kernel/events/core.c:2150
 [<ffffffff8160c36f>] remote_function+0x12f/0x1b0 kernel/events/core.c:74
 [<ffffffff814eab83>] generic_exec_single+0x253/0x450 kernel/smp.c:156
 [<ffffffff814eaf96>] smp_call_function_single+0x216/0x340 kernel/smp.c:300
 [<ffffffff81606b7c>] task_function_call+0x11c/0x130 kernel/events/core.c:101
 [<ffffffff81611bc8>] perf_install_in_context+0x318/0x640
kernel/events/core.c:2193
 [<ffffffff81628a20>] SYSC_perf_event_open+0x1720/0x1fa0
kernel/events/core.c:8457
 [<ffffffff8162fad9>] SyS_perf_event_open+0x39/0x50 kernel/events/core.c:8153
 [<ffffffff85e8da36>] entry_SYSCALL_64_fastpath+0x16/0x7a
arch/x86/entry/entry_64.S:185
---[ end trace 3930c39e9993a16d ]---