linux-kernel - Re: rb tree hrtimer lockup bug (found by perf

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.DEB.2.02.1403191552570.18573@ionos.tec.linutronix.de>
Date:	Wed, 19 Mar 2014 16:05:29 +0100 (CET)
From:	Thomas Gleixner <tglx@...utronix.de>
To:	Vince Weaver <vincent.weaver@...ne.edu>
cc:	linux-kernel@...r.kernel.org, "H. Peter Anvin" <hpa@...or.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...hat.com>
Subject: Re: rb tree hrtimer lockup bug (found by perf_fuzzer)

On Wed, 19 Mar 2014, Vince Weaver wrote:
> On Wed, 19 Mar 2014, Thomas Gleixner wrote:
> 
> > On Wed, 19 Mar 2014, Vince Weaver wrote:
> > > On Tue, 18 Mar 2014, Thomas Gleixner wrote:
> > > your patch didn't seem to print anything additional the first time throug.
> > > 
> > > I then tried the trace command you suggested, but I'm getting an empty
> > > ftrace buffer which possibly means I don't have enough ftrace kernel 
> > > options enabled.
> > > 
> > > Here's the most recent boot crash.
> > > 
> > > [    5.367069] ODEBUG: Info active (active state 0) object type: timer_list hint:           (null)
> > 
> > Stupid me. We get the hint from the wreckaged object ....
> > 
> > A hopefully better approach is the delta patch below.
> 
> with that applied on top:
> 
> [    5.342681] Invalid timer base: tmr ffff880117740150 tmr->base           (null) base ffff880118618000
> [    5.352786] ------------[ cut here ]------------
> [    5.357911] WARNING: CPU: 4 PID: 0 at lib/debugobjects.c:260 debug_print_object+0x8c/0xb0()
> [    5.367023] ODEBUG: Info active (active state 0) object type: timer_list hint:           (null) delayed_work_timer_fn+0x0/0x20

Good, we have at least a point where we can gather useful
information. The timer belongs to a delayed work. The work itself is
tracked via debug objects as well. So we can get that as well. Delta
patch below.

Thanks,

	tglx
------

Index: linux-2.6/lib/debugobjects.c
===================================================================
--- linux-2.6.orig/lib/debugobjects.c
+++ linux-2.6/lib/debugobjects.c
@@ -14,6 +14,7 @@
 #include <linux/debugfs.h>
 #include <linux/slab.h>
 #include <linux/hash.h>
+#include <linux/workqueue.h>
 
 #define ODEBUG_HASH_BITS	14
 #define ODEBUG_HASH_SIZE	(1 << ODEBUG_HASH_BITS)
@@ -400,6 +401,20 @@ void debug_object_info(void *addr, struc
 		debug_print_object(obj, "Info");
 
 	raw_spin_unlock_irqrestore(&db->lock, flags);
+
+	addr = container_of(addr, struct delayed_work, timer);
+
+	db = get_bucket((unsigned long) addr);
+
+	raw_spin_lock_irqsave(&db->lock, flags);
+
+	obj = lookup_object(addr, db);
+	if (!obj)
+		pr_err("Object unknown %p\n", addr);
+	else
+		debug_print_object(obj, "Info");
+
+	raw_spin_unlock_irqrestore(&db->lock, flags);
 }
 
 /**
@@ -452,6 +467,7 @@ int debug_object_activate(void *addr, st
 		}
 		if (!obj->hint)
 			obj->hint = descr->debug_hint ?	descr->debug_hint(addr) : NULL;
+
 		raw_spin_unlock_irqrestore(&db->lock, flags);
 		return ret;
 	}




--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/