linux-kernel - Re: netperf ~50% regression with 2.6.33-rc1, bisect to 1b9508f

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <1264496581.3642.114.camel@minggr.sh.intel.com>
Date:	Tue, 26 Jan 2010 17:03:01 +0800
From:	Lin Ming <ming.m.lin@...el.com>
To:	Mike Galbraith <efault@....de>
Cc:	Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...e.hu>,
	"Zhang, Yanmin" <yanmin_zhang@...ux.intel.com>,
	lkml <linux-kernel@...r.kernel.org>
Subject: Re: netperf ~50% regression with 2.6.33-rc1, bisect to 1b9508f

On Mon, 2010-01-25 at 19:35 +0800, Mike Galbraith wrote:
> On Mon, 2010-01-25 at 18:03 +0800, Lin Ming wrote:
> 
> > With above commit, the idle balance was rate limited, so CPU 15(server,
> > waiting data from client) is idle at most time.
> > 
> > CPU0(client) executes as below,
> > 
> > try_to_wake_up
> >    check_preempt_curr_idle
> >       resched_task
> >          smp_send_reschedule
> > 
> > This causes a lot of rescheduling IPI.
> > 
> > This commit can't be reverted due to conflict, so I just add below code
> > to disable "Rate-limit newidle" and the performance was recovered.
> > 
> > diff --git a/kernel/sched.c b/kernel/sched.c
> > index 18cceee..588fdef 100644
> > --- a/kernel/sched.c
> > +++ b/kernel/sched.c
> > @@ -4421,9 +4421,6 @@ static void idle_balance(int this_cpu, struct rq *this_rq)
> >  
> >  	this_rq->idle_stamp = this_rq->clock;
> >  
> > -	if (this_rq->avg_idle < sysctl_sched_migration_cost)
> > -		return;
> > -
> >  	for_each_domain(this_cpu, sd) {
> >  		unsigned long interval;
> >  
> 
> Heh, so you should see the same thing with newidle disabled, as it was
> in .31 and many kernels prior.  Do you?

Weird.
2.6.31 does not have so many reschedule IPI.

This Nehalem machine has 3 domain levels,
$ grep . cpu0/domain*/name
cpu0/domain0/name:SIBLING
cpu0/domain1/name:MC
cpu0/domain2/name:NODE

For 2.6.31, SD_BALANCE_NEWIDLE is only set on SIBLING level.
For 2.6.32-rc1, SD_BALANCE_NEWIDLE is set on all 3 levels.

I can see many reschedule IPI in 2.6.32-rc1 if SD_BALANCE_NEWIDLE is
cleared for all 3 levels.
But for 2.6.31, I didn't see so many IPI even SD_BALANCE_NEWIDLE is
cleared on SIBLING level.

So it seems something happens between 2.6.31 and 2.6.32-rc1.
I'll bisect ...

Lin Ming

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/