linux-kernel - Re: [ANNOUNCE] Merkey's Kernel Debugger

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 11 Aug 2008 07:11:42 -0600 (MDT)
From:	jmerkey@...fmountaingroup.com
To:	"Vivek Goyal" <vgoyal@...hat.com>
Cc:	"Andi Kleen" <andi@...stfloor.org>,
	"Keith Owens" <kaos@....com.au>, "Jay Lan" <jlan@....com>,
	"Christoph Lameter" <cl@...ux-foundation.org>,
	"Stefan Richter" <stefanr@...6.in-berlin.de>,
	"Nick Piggin" <nickpiggin@...oo.com.au>,
	jmerkey@...fmountaingroup.com,
	"Geert Uytterhoeven" <geert@...ux-m68k.org>,
	"Josh Boyer" <jwboyer@...il.com>, linux-kernel@...r.kernel.org,
	"Takenori Nagano" <t-nagano@...jp.nec.com>,
	"Bernhard Walle" <bwalle@...e.de>
Subject: Re: [ANNOUNCE] Merkey's Kernel Debugger

I found a problem with APIC NMI support which seems to affect all the
debuggers, but appears machine specific -- at least I can reproduce it
with all of the modules MDB, KDB, and KGDB modules on my ACER 2410 dual
core laptop.  It explains the mysterious hangs I would see in KDB all the
time on SMP systems.

The call:

send_IPI_allbutself(vector)

will hard hang an on ACER laptop with dual core processors if issued while
any one of the processors are actively inside an INT 1 handler, then take
a SECOND NMI inside of this path, and nest.   It hangs the requesting
(focus) processor during nested interrupts if a target processor is A)
inside an INT 1 exception B) takes an NMI interrupt C) returns from the
NMI back into the INT1 D) receives a second NMI.

I am aware that a second NMI will not propagate to a processor currently
servicing an NMI until the processor sees an IRET instruction (at least
this is how intel worked years back).

I have not been able to reproduce it on the Xeon based motherboards.  I
have seen the APIC bus hang this way on my other OS project -- when the
APIC was programmed incorrectly, and assume it must be a bug in the APIC,
how the APIC is programmed by Linux, etc.

I am coding around the problem to prevent such convoluted nesting levels
in MDB (this was from testing) but this was the final test for enabling
SSB and all the fixes before I post and rc3 patch series which really
cleanup up the code, and there's a mystery with send_IPI_allbutself().

Jeff

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/