linux-kernel - Re: [BUG] Stack overflow when running perf and function tracer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <874kmcndy5.fsf@nanos.tec.linutronix.de>
Date:   Fri, 30 Oct 2020 12:36:02 +0100
From:   Thomas Gleixner <tglx@...utronix.de>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Steven Rostedt <rostedt@...dmis.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...nel.org>, kan.liang@...ux.intel.com,
        like.xu@...ux.intel.com
Subject: Re: [BUG] Stack overflow when running perf and function tracer

On Fri, Oct 30 2020 at 11:32, Peter Zijlstra wrote:
> On Fri, Oct 30, 2020 at 11:26:01AM +0100, Thomas Gleixner wrote:
>
>> > The only thing I can come up with in a hurry is that that dummy_iregs
>> > thing really should be static. That's 168 bytes of stack out the window
>> > right there.
>> 
>> What's worse is perf_sample_data which is 384 bytes and is 64 bytes
>> aligned.

And there is also x86_perf_regs which is another 176 bytes ....

> Urgh, that thing just keeps on growing :/
>
> I'll try and have a look if we can both shrink the thing and move it off
> stack.

Even then we still need to worry about the stack depth because that
tracer code was not yet done. Throw some BPF at it and it goes south
again.

So the real question is what else is on that stack which blows it up
close to 4k? Btw, it would be massively helpful for this kind of crash
to print the actual stack depth per entry in the backtrace.

Here is the partial stack trace:
                                                Stack usage
  ring_buffer_lock_reserve+0x12c/0x380          
  trace_function+0x27/0x130
  function_trace_call+0x133/0x180
  perf_output_begin+0x4d/0x2d0                   64+
  perf_log_throttle+0x9a/0x120                  470+
  __perf_event_account_interrupt+0xa9/0x120
  __perf_event_overflow+0x2b/0xf0               
  __intel_pmu_pebs_event+0x2ec/0x3e0            760+
  intel_pmu_drain_pebs_nhm+0x268/0x330          200+
  handle_pmi_common+0xc2/0x2b0                  

The missing call chain is:

 intel_pmu_handle_irq or intel_pmu_handle_irq_v4
 perf_event_nmi_handler
 nmi
 
So the larger offenders accumulate to ~1.5k data on stack, but
where is the rest of the 4k?

It's about 15 calls. So to fill up the stack you'd need about 230 bytes
per call to go up to 4k.

Something is fishy.

Thanks,

        tglx