linux-kernel - Re: [.32-rc3] scheduler: iwlagn consistently high in "waiting for CPU"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20091008064041.67219b13@infradead.org>
Date:	Thu, 8 Oct 2009 06:40:41 -0700
From:	Arjan van de Ven <arjan@...radead.org>
To:	Mike Galbraith <efault@....de>
Cc:	Frans Pop <elendil@...net.nl>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Ingo Molnar <mingo@...e.hu>,
	Peter Zijlstra <peterz@...radead.org>,
	linux-wireless@...r.kernel.org
Subject: Re: [.32-rc3] scheduler: iwlagn consistently high in "waiting for
 CPU"

On Thu, 08 Oct 2009 06:05:43 +0200
Mike Galbraith <efault@....de> wrote:

> On Wed, 2009-10-07 at 20:34 +0200, Frans Pop wrote:
> > On Wednesday 07 October 2009, Frans Pop wrote:
> > > On Tuesday 06 October 2009, Frans Pop wrote:
> > > > I've checked for 2.6.31.1 now and iwlagn is listed high there
> > > > too when the system is idle, but with normal values of 60-100
> > > > ms. And phy0 has normal values of below 10 ms.
> > > > I've now rebooted with today's mainline git; phy0 now
> > > > frequently shows with values of around 100 ms too (i.e. higher
> > > > than last time).
> > >
> > > Mike privately sent me a script to try to capture the latencies
> > > with perf, but the perf output does not show any high latencies
> > > at all. It looks as if we may have found a bug in latencytop here
> > > instead.
> > 
> > Not sure if it's relevant nor what it means, but I frequently see
> > two lines for iwlagn, e.g:
> > 
> >     Scheduler: waiting for cpu              102.4 msec         99.7
> > % .                                         3.3 msec          0.3 %
> > 
> > I get the same results with both latencytop 0.4 and 0.5.
> 
> OK, I see latencytop spikes here on an idle box too, to the tune of up
> to a _second_.  Booting with nohz=off seems to have cured it.

interesting..


by chance, does this patch fix it ?

>From 33725d4939f457b12d7bc1bcbcc0dfb8b2f5bd48 Mon Sep 17 00:00:00 2001
From: Arjan van de Ven <arjan@...ux.intel.com>
Date: Thu, 24 Sep 2009 13:24:16 +0200
Subject: [PATCH] x86, timers: check for pending timers after (device) interrupts

Now that range timers and deferred timers are common, I found a
problem with these using the "perf timechart" tool.

It turns out that on x86, these two 'opportunistic' timers only
get checked when another "real" timer happens.
These opportunistic timers have the objective to save power by
hitchhiking on other wakeups, as to avoid CPU wakeups by themselves
as much as possible.

The change in this patch runs this check not only at timer interrupts,
but at all (device) interrupts. The effect is that
1) the deferred timers/range timers get delayed less
2) the range timers cause less wakeups by themselves because
   the percentage of hitchhiking on existing wakeup events goes up.

I've verified the working of the patch using "perf timechart",
the original exposed bug is gone with this patch.

Signed-off-by: Arjan van de Ven <arjan@...ux.intel.com>
---
 arch/x86/kernel/irq.c |    2 ++
 arch/x86/kernel/smp.c |    1 +
 2 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c
index 74656d1..3912061 100644
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -244,6 +244,7 @@ unsigned int __irq_entry do_IRQ(struct pt_regs *regs)
 				__func__, smp_processor_id(), vector, irq);
 	}
 
+	run_local_timers();
 	irq_exit();
 
 	set_irq_regs(old_regs);
@@ -268,6 +269,7 @@ void smp_generic_interrupt(struct pt_regs *regs)
 	if (generic_interrupt_extension)
 		generic_interrupt_extension();
 
+	run_local_timers();
 	irq_exit();
 
 	set_irq_regs(old_regs);
diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c
index ec1de97..d915d95 100644
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -198,6 +198,7 @@ void smp_reschedule_interrupt(struct pt_regs *regs)
 {
 	ack_APIC_irq();
 	inc_irq_stat(irq_resched_count);
+	run_local_timers();
 	/*
 	 * KVM uses this interrupt to force a cpu out of guest mode
 	 */
-- 
1.6.0.6


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/