linux-kernel - Re: perf: WARNING perfevents: irq loop stuck!

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150508075503.GC5403@gmail.com>
Date:	Fri, 8 May 2015 09:55:03 +0200
From:	Ingo Molnar <mingo@...nel.org>
To:	Vince Weaver <vincent.weaver@...ne.edu>
Cc:	linux-kernel@...r.kernel.org,
	Peter Zijlstra <peterz@...radead.org>,
	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Jiri Olsa <jolsa@...hat.com>, Ingo Molnar <mingo@...hat.com>,
	Paul Mackerras <paulus@...ba.org>
Subject: Re: perf: WARNING perfevents: irq loop stuck!


* Ingo Molnar <mingo@...nel.org> wrote:

> 
> * Vince Weaver <vincent.weaver@...ne.edu> wrote:
> 
> > So this is just a warning, and I've reported it before, but the 
> > perf_fuzzer triggers this fairly regularly on my Haswell system.
> > 
> > It looks like fixed counter 0 (retired instructions) being set to 
> > 0000fffffffffffe occasionally causes an irq loop storm and gets 
> > stuck until the PMU state is cleared.
> 
> So 0000fffffffffffe corresponds to 2 events left until overflow, 
> right? And on Haswell we don't set x86_pmu.limit_period AFAICS, so we 
> allow these super short periods.
> 
> Maybe like on Broadwell we need a quirk on Nehalem/Haswell as well, 
> one similar to bdw_limit_period()? Something like the patch below?
> 
> Totally untested and such. I picked 128 because of Broadwell, but 
> lower values might work as well. You could try to increase it to 3 and 
> upwards and see which one stops triggering stuck NMI loops?
> 
> Thanks,
> 
> 	Ingo
> 
> Signed-off-by: Ingo Molnar <mingo@...nel.org>
> 
> ---
>  arch/x86/kernel/cpu/perf_event_intel.c | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/cpu/perf_event_intel.c b/arch/x86/kernel/cpu/perf_event_intel.c
> index 960e85de13fb..26b13ea8299c 100644
> --- a/arch/x86/kernel/cpu/perf_event_intel.c
> +++ b/arch/x86/kernel/cpu/perf_event_intel.c
> @@ -2479,6 +2479,15 @@ hsw_get_event_constraints(struct cpu_hw_events *cpuc, int idx,
>  
>  	return c;
>  }
> +/*
> + * Really short periods might create infinite PMC NMI loops on Haswell,
> + * so limit them to 128. There's no official erratum for this AFAIK.
> + */
> +static unsigned int hsw_limit_period(struct perf_event *event, unsigned int left)
> +{
> +	return max(left, 128U);
> +}
> +
>  
>  /*
>   * Broadwell:
> @@ -2495,7 +2504,7 @@ hsw_get_event_constraints(struct cpu_hw_events *cpuc, int idx,
>   * Therefore the effective (average) period matches the requested period,
>   * despite coarser hardware granularity.
>   */
> -static unsigned bdw_limit_period(struct perf_event *event, unsigned left)
> +static unsigned int bdw_limit_period(struct perf_event *event, unsigned left)
>  {
>  	if ((event->hw.config & INTEL_ARCH_EVENT_MASK) ==
>  			X86_CONFIG(.event=0xc0, .umask=0x01)) {
> @@ -3265,6 +3274,7 @@ __init int intel_pmu_init(void)
>  		x86_pmu.hw_config = hsw_hw_config;
>  		x86_pmu.get_event_constraints = hsw_get_event_constraints;
>  		x86_pmu.cpu_events = hsw_events_attrs;
> +		x86_pmu.limit_period = hsw_limit_period;
>  		x86_pmu.lbr_double_abort = true;
>  		pr_cont("Haswell events, ");
>  		break;

Also, I'd apply the quirk not just to Haswell, but Nehalem, Westmere 
and Ivy Bridge as well, I have seen it as early as on a Nehalem 
prototype box.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/