linux-kernel - Re: [PATCH 3/4] kgdb,clocksource: Prevent kernel hang in kernel debugger

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LFD.2.00.1001261108410.5926@localhost.localdomain>
Date:	Tue, 26 Jan 2010 11:37:18 +0100 (CET)
From:	Thomas Gleixner <tglx@...utronix.de>
To:	Dongdong Deng <libfetion@...il.com>
cc:	Martin Schwidefsky <schwidefsky@...ibm.com>,
	Jason Wessel <jason.wessel@...driver.com>,
	linux-kernel@...r.kernel.org, kgdb-bugreport@...ts.sourceforge.net,
	mingo@...e.hu, John Stultz <johnstul@...ibm.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Magnus Damm <damm@...l.co.jp>
Subject: Re: [PATCH 3/4] kgdb,clocksource: Prevent kernel hang in kernel
 debugger

On Tue, 26 Jan 2010, Dongdong Deng wrote:
> On Tue, Jan 26, 2010 at 4:50 PM, Thomas Gleixner <tglx@...utronix.de> wrote:
> > On Tue, 26 Jan 2010, Martin Schwidefsky wrote:
> >> On Mon, 25 Jan 2010 22:26:39 -0600
> >> Jason Wessel <jason.wessel@...driver.com> wrote:
> >>
> >> > This is a regression fix against: 0f8e8ef7c204988246da5a42d576b7fa5277a8e4
> >> >
> >> > Spin locks were added to the clocksource_resume_watchdog() which cause
> >> > the kernel debugger to deadlock on an SMP system frequently.
> >> >
> >> > The kernel debugger can try for the lock, but if it fails it should
> >> > continue to touch the clocksource watchdog anyway, else it will trip
> >> > if the general kernel execution has been paused for too long.
> >> >
> >> > This introduces an possible race condition where the kernel debugger
> >> > might not process the list correctly if a clocksource is being added
> >> > or removed at the time of this call.  This race is sufficiently rare vs
> >> > having the kernel debugger hang the kernel
> >> >
> >> > CC: Thomas Gleixner <tglx@...utronix.de>
> >> > CC: Martin Schwidefsky <schwidefsky@...ibm.com>
> >> > CC: John Stultz <johnstul@...ibm.com>
> >> > CC: Andrew Morton <akpm@...ux-foundation.org>
> >> > CC: Magnus Damm <damm@...l.co.jp>
> >> > Signed-off-by: Jason Wessel <jason.wessel@...driver.com>
> >>
> >> The first question I would ask is why does the kernel deadlock? Can we
> >> have a backchain of a deadlock please?
> >
> > The problem arises when the kernel is stopped inside the watchdog code
> > with watchdog_lock held. When kgdb restarts execution then it touches
> > the watchdog to avoid that TSC gets marked unstable.
> >
> >> Hmm, there are all kinds of races if the watchdog code gets interrupted
> >> by the kernel debugger. Wouldn't it be better to just disable the
> >> watchdog while the kernel debugger is active?
> >
> > No, we can keep it and in most cases it clocksource_touch_watchdog()
> > helps to keep TSC alive. A simple "if (!trylock) return;" should solve
> > the deadlock problem for kgdb without opening a can of worms.
> 
> Is it possible that we reset the clocksource watchdog during in
> clocksource_watchdog() ?
> 
> >From the code view, The action of reset clocksource watchdog is just
> set the CLOCK_SOURCE_WATCHDOG flag.
> thus if we reset it before using, I think the logic will be right.

No, it's not. It just brings back the old flag based logic which we
removed.

The correct way to solve this is a documented 

    if (!trylock())
       return;

in clocksource_touch_watchdog(). And that's what I'm going to push
linuswards.

There is no sane way to reliably prevent TSC from becoming unstable
when kgdb stops the kernel inside the watchdog code. And I do not care
about that at all.

I'm not going to clutter code with crazy workarounds just because some
people believe that using a kernel debugger is a good idea. If people
insist on using kgdb then the possible "TSC becomes unstable" side
effect is the least of their problems.

Thanks,

	tglx