linux-kernel - Re: [PATCH] perfcounters: fix "perf counters kills oprofile" bug

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090205142607.GC28443@elte.hu>
Date:	Thu, 5 Feb 2009 15:26:07 +0100
From:	Ingo Molnar <mingo@...e.hu>
To:	Mike Galbraith <efault@....de>
Cc:	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Arjan van de Ven <arjan@...radead.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Paul Mackerras <paulus@...ba.org>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] perfcounters: fix "perf counters kills oprofile" bug


* Mike Galbraith <efault@....de> wrote:

> Impact: fix "perf counters kills oprofile" bug
> 
> Both oprofile and perfcounters register an NMI die handler, but only one
> can handle the NMI.  Conveniently, oprofile unregisters it's notifier
> when not actively in use, so setting it's notifier priority higher than
> perfcounter's allows oprofile to borrow the NMI for the duration of it's
> run.  Tested/works both as module and built-in.
> 
> While testing, I found that if kerneltop was generating NMIs at very
> high frequency, the kernel may panic when oprofile registered it's
> handler.  This turned out to be because oprofile registers it's handler
> before reset_value has been allocated, so if an NMI comes in while it's
> still setting up, kabOom.  Rather than try more invasive changes, I
> followed the lead of other places in op_model_ppro.c, and simply
> returned in that highly unlikely event.  (debug warnings attached)

applied to tip:perfcounters/core, thanks Mike!

> I can break this into two patches if you prefer, but since the panic was 
> initiated by borrowing the active NMI, I figured they belong together.

No need, it's fine this way. Note that there's two commits from you: i 
applied your earier fix of this already, and now i did a delta patch of the 
other bug you found. The delta patch i have applied is below.

Thanks,

	Ingo

--------------------->
>From 82aa9a1829199233f9bdaf26e2ee271114f4701e Mon Sep 17 00:00:00 2001
From: Ingo Molnar <mingo@...e.hu>
Date: Thu, 5 Feb 2009 15:23:08 +0100
Subject: [PATCH] perfcounters: fix "perf counters kills oprofile" bug, v2

Impact: fix kernel crash

Both oprofile and perfcounters register an NMI die handler, but only one
can handle the NMI.  Conveniently, oprofile unregisters it's notifier
when not actively in use, so setting it's notifier priority higher than
perfcounter's allows oprofile to borrow the NMI for the duration of it's
run.  Tested/works both as module and built-in.

While testing, I found that if kerneltop was generating NMIs at very
high frequency, the kernel may panic when oprofile registered it's
handler.  This turned out to be because oprofile registers it's handler
before reset_value has been allocated, so if an NMI comes in while it's
still setting up, kabOom.  Rather than try more invasive changes, I
followed the lead of other places in op_model_ppro.c, and simply
returned in that highly unlikely event.  (debug warnings attached)

Signed-off-by: Mike Galbraith <efault@....de>
Signed-off-by: Ingo Molnar <mingo@...e.hu>
---
 arch/x86/oprofile/op_model_ppro.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/arch/x86/oprofile/op_model_ppro.c b/arch/x86/oprofile/op_model_ppro.c
index 07c9145..85eb626 100644
--- a/arch/x86/oprofile/op_model_ppro.c
+++ b/arch/x86/oprofile/op_model_ppro.c
@@ -126,6 +126,13 @@ static int ppro_check_ctrs(struct pt_regs * const regs,
 	u64 val;
 	int i;
 
+	/*
+	 * This can happen if perf counters are in use when
+	 * we steal the die notifier NMI.
+	 */
+	if (unlikely(!reset_value))
+		goto out;
+
 	for (i = 0 ; i < num_counters; ++i) {
 		if (!reset_value[i])
 			continue;
@@ -136,6 +143,7 @@ static int ppro_check_ctrs(struct pt_regs * const regs,
 		}
 	}
 
+out:
 	/* Only P6 based Pentium M need to re-unmask the apic vector but it
 	 * doesn't hurt other P6 variant */
 	apic_write(APIC_LVTPC, apic_read(APIC_LVTPC) & ~APIC_LVT_MASKED);
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/