linux-kernel - Re: rb tree hrtimer lockup bug (found by perf

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 31 Mar 2014 13:46:45 +0200
From:	Ingo Molnar <mingo@...nel.org>
To:	Thomas Gleixner <tglx@...utronix.de>
Cc:	Vince Weaver <vincent.weaver@...ne.edu>,
	LKML <linux-kernel@...r.kernel.org>,
	"H. Peter Anvin" <hpa@...or.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...hat.com>, Greg KH <greg@...ah.com>
Subject: Re: rb tree hrtimer lockup bug (found by perf_fuzzer)


* Thomas Gleixner <tglx@...utronix.de> wrote:

> On Thu, 27 Mar 2014, Vince Weaver wrote:
> > On Wed, 26 Mar 2014, Thomas Gleixner wrote:
> > > Ok. So we know now what we are looking for.
> > > 
> > > [    1.579996] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> > > ÿ[    1.607279] 00:09: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
> > > [    1.615032] kobject: 'ttyS1' (ffff88011772ac10): kobject_release, parent           (null) (delayed 250)
> > > [    1.624534] kobject: '(null)' (ffff8801177400f0): kobject_release, parent           (null) (delayed 500)
> > > [    1.654213] 0000:00:16.3: ttyS1 at I/O 0xf0e0 (irq = 19, base_baud = 115200) is a 16550A
> > > 
> > > [    3.294047] Invalid timer base: tmr ffff880117740150 tmr->base           (null) base ffff880118898000
> > > 
> > > 1634110us : obj: ffff880117740130 initialized kobject_delayed_cleanup+0x0/0x90
> > > 
> > > So that happens in the context of the 8250 serial driver.
> > > 
> > > ...
> > > 
> > > Below is a patch which gives us the call path of the unnamed object
> > > which causes the crash.
> > 
> > I've attached the boot log with that patch applied.
> 
> Vince, can you please disable CONFIG_DEBUG_KOBJECT_RELEASE and remove
> all the debug patches to see whether the issue goes away?
> 
> I had a deeper look down that code path and the issue is, that the
> serial core is not compatible with the deferred kobject release.
> 
> The tty_io layer uses a kobject embedded in its internal tty device
> representation and reuses that.
> 
> So it seems that for whatever reason the tty layer releases ttyS1 and
> then initializes it again. So the deferred release will queue the
> object for release while the tty layer happily reinitializes it.
> 
> See tty_register_device() and tty_unregister.
> 
> Greg?

Hm, this reminds me of the following randconfig testing patch I've 
been carrying for some time:

================>
$ cat patches/qa-dont-boot-DEBUG_KOBJECT_RELEASE.patch
Subject: qa: dont boot DEBUG_KOBJECT_RELEASE
From: Ingo Molnar <mingo@...nel.org>
Date: Mon Oct 7 16:15:46 CEST 2013

Signed-off-by: Ingo Molnar <mingo@...nel.org>
---
 lib/Kconfig.debug |    3 +++
 1 file changed, 3 insertions(+)

Index: linux2/lib/Kconfig.debug
===================================================================
--- linux2.orig/lib/Kconfig.debug
+++ linux2/lib/Kconfig.debug
@@ -1025,6 +1025,9 @@ config DEBUG_KOBJECT
 config DEBUG_KOBJECT_RELEASE
        bool "kobject release debugging"
        depends on DEBUG_OBJECTS_TIMERS
+       # Frequent, hard to debug crashes:
+       depends on BROKEN_BOOT_ALLOWED
+       select BROKEN_BOOT
        help
          kobjects are reference counted objects.  This means that their
          last reference count put is not predictable, and the kobject can

I never fully tracked it down and forgot about it.

Thanks,

		Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/