linux-kernel - Re: [PATCH 21/21] tracing: Add optional percpu buffers for trace

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1316777306.9084.11.camel@twins>
Date:	Fri, 23 Sep 2011 13:28:26 +0200
From:	Peter Zijlstra <peterz@...radead.org>
To:	Steven Rostedt <rostedt@...dmis.org>
Cc:	linux-kernel@...r.kernel.org, Ingo Molnar <mingo@...e.hu>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH 21/21] tracing: Add optional percpu buffers for
 trace_printk()

On Fri, 2011-09-23 at 07:16 -0400, Steven Rostedt wrote:
> On Fri, 2011-09-23 at 13:07 +0200, Peter Zijlstra wrote:
> > On Fri, 2011-09-23 at 13:02 +0200, Peter Zijlstra wrote:
> > > On Thu, 2011-09-22 at 18:09 -0400, Steven Rostedt wrote:
> > > > 
> > > > Currently, trace_printk() uses a single buffer to write into
> > > > to calculate the size and format needed to save the trace. To
> > > > do this safely in an SMP environment, a spin_lock() is taken
> > > > to only allow one writer at a time to the buffer. But this could
> > > > also affect what is being traced, and add synchronization that
> > > > would not be there otherwise. 
> > > 
> > > so trace_printk() isn't NMI safe? #$%@^%@@$%@
> 
> It is NMI safe, always was (I use it there too). It has a percpu
> recursion detection (always has), thus if an NMI interrupts a current
> trace_printk(), the NMI trace_printk() will not print. I could add an
> NMI buffer to allow NMIs to print, but so far, we don't usually have
> issues with trace_printk(). Heck, I'm not sure printk() wont cause
> issues in NMIs. I think trace_printk() is still safer than printk.

Of course, printk() is most useless, its too slow, and its definitely
not NMI-safe.

If you've only got a single buffer, I don't see why you would need
per-cpu recursion detection, just spin_try_lock() the thing and if you
fail, bail. But that's not what the changelog said, or at least implied.

The possibility of dropping trace output like that is most worrying
though. I can imagine myself tearing my hair out trying to make sense of
a trace, and then wanting to kick you after finding out it lost the
crucial bit.

> > better to make all of trace_printk() depend on that extra config, there
> > is absolutely 0 point in having a broken and fully serialized trace
> > 'fail^wfeature'.
> 
> Not, having per cpu buffers still doesn't allow NMIs to interrupt
> trace_printk(). Otherwise the NMI would just corrupt the current percpu
> buffer.

Multiple buffers sounds about right, not sure if you disable interrupts
over the normal trace path, but ISTR you don't, so you need
task/softirq/irq/nmi buffers per cpu.

Really loosing trace output is not an option, ever (aside from stuff
falling of the end of the buffer). Reliability first, performance
second.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/