linux-kernel - Re: [PATCH] tracing: use raw spinlocks instead of spinlocks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20081103064647.GA4826@elte.hu>
Date:	Mon, 3 Nov 2008 07:46:47 +0100
From:	Ingo Molnar <mingo@...e.hu>
To:	Steven Rostedt <rostedt@...dmis.org>
Cc:	Frederic Weisbecker <fweisbec@...il.com>,
	Linux Kernel <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] tracing: use raw spinlocks instead of spinlocks


* Steven Rostedt <rostedt@...dmis.org> wrote:

> On Mon, 3 Nov 2008, Frederic Weisbecker wrote:
> 
> > When I tried to figure out why my experimental function's return 
> > tracer was hanging, I discovered that it was partially caused by 
> > the fact that the ring buffer might use the usual spinlocks during 
> > entry insertion.
> > 
> > ring_buffer_lock_reserve() -> rb_reserve_next_event() -> 
> > __rb_reserve_next() -> spin_lock_irqsave()
> > 
> > Since this last function is traced, the result is a recursion 
> > during the trace. I guess it happens too with the function tracer.
> > 
> > We should use the raw_spin_locks which are not traced.
> > 
> > Signed-off-by: Frederic Weisbecker <fweisbec@...il.com>
> 
> Frederic,
> 
> Ingo has been very adamant about not using raw_spin_locks in the 
> ring buffers. My original code did this, and he nacked it. The 
> reason being (and he eventually convinced me) was that by using raw, 
> we not only do not trace the locking, we also remove the lock 
> checking. This code can easily produce deadlocks, so we do not want 
> the lock checking removed.
> 
> The real fix is to find a way in your tracer to detect the 
> recursion, and be able to prevent it. Like the atomic disables I use 
> in ftrace. It does the same thing. It leaves the lockdep checking on 
> its own locks, but can also detect if the lock checking caused it to 
> recurse. When the recusion is detected, the tracer itself will not 
> trace.

i'm wondering, does the changing to raw-spinlocks fix the deadlock? 
It's generally just the lack of recursion checking that is causing 
lockdep troubles - and recursion checking we want for all the more 
intrusive ftrace plugins anyway.

Frederic, do you have trouble finding the source of the deadlock? In 
theory the NMI watchdog should be able to punch through it. (if not 
then we need to improve things so that it can)

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/