linux-kernel - Re: perf: fuzzer KASAN perf_callchain

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Wed, 16 Nov 2016 17:38:06 +0100
From:   Dmitry Vyukov <dvyukov@...gle.com>
To:     Vince Weaver <vincent.weaver@...ne.edu>
Cc:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        "davej@...emonkey.org.uk" <davej@...emonkey.org.uk>,
        Stephane Eranian <eranian@...il.com>
Subject: Re: perf: fuzzer KASAN perf_callchain_store on amd

On Wed, Nov 16, 2016 at 5:33 PM, Vince Weaver <vincent.weaver@...ne.edu> wrote:
>
> Possibly related to the other reports, I'm getting this on the AMD a10
> machine.  I don't have the earliest trigger for this because my testing
> setup is poorly designed so the haswell machine crashing the ethernet
> switch caused the serial port logs to be lost.
>
> It turns out the framepointer wasn't enabled on this machine, I'm
> re-enabling and I'll see if I can reproduce.
>
> As an aside, it might be random chance, but I am noticing
> "perf_event_output_backward" is involved in a lot of these
> traces.
>
> [118724.973843] BAD LUCK: lost 45131 message(s) from NMI context!
> [118724.973845] ==================================================================
> [118724.988303] BUG: KASAN: slab-out-of-bounds in perf_callchain_store+0x69/0x84 at addr ffff8801d4fbe800
> [118724.998335] Write of size 8 by task perf_fuzzer/17994
> [118725.004205] CPU: 0 PID: 17994 Comm: perf_fuzzer Tainted: G    B   W    L  4.9.0-rc5+ #39
> [118725.013189] Hardware name: Hewlett-Packard HP Compaq Pro 6305 SFF/1850, BIOS K06 v02.57 08/16/2013
> [118725.023108]  0000000000000000^Ac ffffffff813a8d66^Ac ffff8801d4fbf700^Ac ffff8801ed800a00^Ac
> [118725.032198]  ffffffff811d229c^Ac ffff8801d4fbd700^Ac 1ffff1003a9f7d00^Ac ffffed003a9f7d00^Ac
> [118725.041297]  ffffffff811d263e^Ac 0000000000000096^Ac ffff8801eabb7d30^Ac ffff8801edc0ba88^Ac
> [118725.050433] Call Trace:
> [118725.053940]  <NMI>  [<ffffffff813a8d66>] ? dump_stack+0x46/0x59
> [118725.061001]  [<ffffffff811d229c>] ? kasan_object_err+0x17/0x6b
> [118725.068017]  [<ffffffff811d263e>] ? kasan_report+0x2c0/0x41a
> [118725.074880]  [<ffffffff810f490d>] ? __module_text_address+0xc/0x86
> [118725.082302]  [<ffffffff81067d7f>] ? copy_process.part.40+0x12d/0x2789
> [118725.090027]  [<ffffffff810032bc>] ? perf_callchain_store+0x69/0x84
> [118725.097519]  [<ffffffff810063da>] ? perf_callchain_kernel+0xdd/0xf7
> [118725.105117]  [<ffffffff8116aab6>] ? get_perf_callchain+0x1ad/0x2af
> [118725.112667]  [<ffffffff8116ac62>] ? perf_callchain+0xaa/0xb5
> [118725.119719]  [<ffffffff8108c78a>] ? __kernel_text_address+0x1/0x3d
> [118725.127333]  [<ffffffff81166785>] ? perf_prepare_sample+0xd8/0x5c0
> [118725.134977]  [<ffffffff810062dc>] ? arch_perf_update_userpage+0x104/0x125
> [118725.143273]  [<ffffffff81166cdb>] ? perf_event_output_backward+0x1a/0x54
> [118725.151511]  [<ffffffff81163a48>] ? __perf_event_overflow+0x188/0x222
> [118725.159528]  [<ffffffff81005b60>] ? x86_pmu_handle_irq+0x147/0x184
> [118725.167321]  [<ffffffff8108c78a>] ? __kernel_text_address+0x1/0x3d
> [118725.175144]  [<ffffffff810094af>] ? perf_ibs_handle_irq+0x54c/0x54c
> [118725.183086]  [<ffffffff81024cdb>] ? perf_trace_nmi_handler+0x123/0x14a
> [118725.191319]  [<ffffffff8102a0fe>] ? cycles_2_ns+0x5c/0xe4
> [118725.198452]  [<ffffffff8102a0fe>] ? cycles_2_ns+0x5c/0xe4
> [118725.205588]  [<ffffffff81003efd>] ? perf_event_nmi_handler+0x22/0x39
> [118725.213722]  [<ffffffff81003efd>] ? perf_event_nmi_handler+0x22/0x39
> [118725.221856]  [<ffffffff8102520c>] ? nmi_handle+0x62/0x153
> [118725.229057]  [<ffffffff810094af>] ? perf_ibs_handle_irq+0x54c/0x54c
> [118725.237169]  [<ffffffff81024bb8>] ? local_touch_nmi+0xd/0xd
> [118725.244619]  [<ffffffff810254e3>] ? default_do_nmi+0x55/0x101
> [118725.252262]  [<ffffffff8102562d>] ? do_nmi+0x9e/0x10f
> [118725.259234]  [<ffffffff816cbb87>] ? end_repeat_nmi+0x1a/0x1e
> [118725.266843]  [<ffffffff810536d3>] ? unwind_next_frame+0x26/0xa7
> [118725.274746]  [<ffffffff8108c752>] ? core_kernel_text+0x29/0x48
> [118725.282588]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.289936]  [<ffffffff8108c78a>] ? __kernel_text_address+0x1/0x3d
> [118725.298209]  [<ffffffff8108c78a>] ? __kernel_text_address+0x1/0x3d
> [118725.306469]  [<ffffffff8108c752>] ? core_kernel_text+0x29/0x48
> [118725.314414]  [<ffffffff8108c78a>] ? __kernel_text_address+0x1/0x3d
> [118725.322728]  <EOE>  <IRQ>  [<ffffffff810536dc>] ? unwind_next_frame+0x2f/0xa7
> [118725.332078]  [<ffffffff810316aa>] ? __save_stack_trace+0xab/0xba
> [118725.340327]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.347870]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.355340]  [<ffffffff811d157c>] ? save_stack+0x9d/0xa6
> [118725.362749]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.370065]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.377344]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.384532]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.391641]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.398711]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.405740]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.412698]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.419610]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.426474]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.433327]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.440135]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.446910]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.453654]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.460383]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.467072]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.473730]  [<ffffffff81168e39>] ? perf_output_copy+0x58/0xf1
> [118725.480913]  [<ffffffff81168b51>] ? perf_output_put_handle+0x46/0xa0
> [118725.488625]  [<ffffffff811635f5>] ? perf_log_throttle+0xfa/0x10c
> [118725.495964]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.502598]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.509193]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.515754]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.522282]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.528779]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.535247]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.541679]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.548113]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.554508]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.560899]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.567254]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.573573]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.579862]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.586132]  [<ffffffff811d1aa8>] ? kasan_unpoison_shadow+0xf/0x2e
> [118725.593285]  [<ffffffff811d1bae>] ? kasan_kmalloc+0x8b/0x9a
> [118725.599818]  [<ffffffff811ce5de>] ? slab_post_alloc_hook+0x31/0x3c
> [118725.606966]  [<ffffffff811cf827>] ? kmem_cache_alloc+0xc6/0x145
> [118725.613851]  [<ffffffff81078994>] ? __sigqueue_alloc+0x5a/0x152
> [118725.620734]  [<ffffffff8107aa8d>] ? __send_signal+0x105/0x30b
> [118725.627428]  [<ffffffff8107b9d5>] ? do_send_sig_info+0x3d/0x73
> [118725.634241]  [<ffffffff811f88f6>] ? send_sigio_to_task+0xb6/0xe4
> [118725.641230]  [<ffffffff8115f24c>] ? perf_pmu_enable+0x2f/0x3d
> [118725.647962]  [<ffffffff810e03f3>] ? task_cputime_zero+0x2c/0x3a
> [118725.654837]  [<ffffffff810e1fab>] ? run_posix_cpu_timers+0xd8/0x687
> [118725.662038]  [<ffffffff810a94e2>] ? nohz_balance_exit_idle+0x36/0x81
> [118725.669327]  [<ffffffff810d46e4>] ? rcu_accelerate_cbs+0x1da/0x39a
> [118725.676481]  [<ffffffff810d2630>] ? rcu_report_qs_rnp+0x77/0x18b
> [118725.683485]  [<ffffffff810d2c93>] ? cpu_needs_another_gp+0xbb/0x11a
> [118725.690771]  [<ffffffff811f9068>] ? send_sigio+0xb6/0x10c
> [118725.697215]  [<ffffffff811f915c>] ? kill_fasync+0x9e/0xdd
> [118725.703673]  [<ffffffff811633c7>] ? perf_event_wakeup+0x6e/0xd6
> [118725.710695]  [<ffffffff81167cf5>] ? perf_pending_event+0x70/0x8a
> [118725.717830]  [<ffffffff8114b569>] ? irq_work_run_list+0x66/0x84
> [118725.724905]  [<ffffffff8114b59b>] ? irq_work_run+0x14/0x29
> [118725.731563]  [<ffffffff81026452>] ? smp_irq_work_interrupt+0x11/0x16
> [118725.739134]  [<ffffffff816cc90f>] ? irq_work_interrupt+0x7f/0x90
> [118725.746386]  <EOI>  [<ffffffff813b3b9d>] ? memcmp+0x1d/0x44
> [118725.753246]  [<ffffffff811d1a57>] ? __asan_load2+0x64/0x64
> [118725.760055]  [<ffffffff813b3ba8>] ? memcmp+0x28/0x44
> [118725.766368]  [<ffffffff813e3101>] ? find_stack+0x3b/0x54
> [118725.773053]  [<ffffffff813e32a6>] ? depot_save_stack+0x136/0x375
> [118725.780468]  [<ffffffff811d157c>] ? save_stack+0x9d/0xa6
> [118725.787218]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.793967]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.800690]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> [118725.807393]  [<ffffffff811d1512>] ? save_stack+0x33/0xa6
> ...


This is heap OOB rather than stack OOB.
Is there an allocation stack/object size/shadow in the report? It
would greatly help debugging.