linux-kernel - Re: sched: Improve load balancing in the presence of idle CPUs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 01 Apr 2015 20:30:34 -0700
From:	Jason Low <jason.low2@...com>
To:	Morten Rasmussen <morten.rasmussen@....com>
Cc:	Preeti U Murthy <preeti@...ux.vnet.ibm.com>,
	"peterz@...radead.org" <peterz@...radead.org>,
	"mingo@...nel.org" <mingo@...nel.org>,
	Daniel Lezcano <daniel.lezcano@...aro.org>,
	"riel@...hat.com" <riel@...hat.com>,
	"vincent.guittot@...aro.org" <vincent.guittot@...aro.org>,
	"srikar@...ux.vnet.ibm.com" <srikar@...ux.vnet.ibm.com>,
	"pjt@...gle.com" <pjt@...gle.com>,
	"benh@...nel.crashing.org" <benh@...nel.crashing.org>,
	"efault@....de" <efault@....de>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"iamjoonsoo.kim@....com" <iamjoonsoo.kim@....com>,
	"svaidy@...ux.vnet.ibm.com" <svaidy@...ux.vnet.ibm.com>,
	"tim.c.chen@...ux.intel.com" <tim.c.chen@...ux.intel.com>,
	jason.low2@...com
Subject: Re: sched: Improve load balancing in the presence of idle CPUs

On Wed, 2015-04-01 at 18:04 +0100, Morten Rasmussen wrote:
> On Wed, Apr 01, 2015 at 07:49:56AM +0100, Preeti U Murthy wrote:
> > 
> > On 04/01/2015 12:24 AM, Jason Low wrote:
> > > On Tue, 2015-03-31 at 14:07 +0530, Preeti U Murthy wrote:
> > >> Hi Jason,
> > >>
> > >> On 03/31/2015 12:25 AM, Jason Low wrote:
> > >>> Hi Preeti,
> > >>>
> > >>> I noticed that another commit 4a725627f21d converted the check in
> > >>> nohz_kick_needed() from idle_cpu() to rq->idle_balance, causing a
> > >>> potentially outdated value to be used if this cpu is able to pull tasks
> > >>> using rebalance_domains(), and nohz_kick_needed() directly returning
> > >>> false.
> > >>
> > >> I see that rebalance_domains() will be run at the end of the scheduler
> > >> tick interrupt handling. trigger_load_balance() only sets the softirq,
> > >> it does not call rebalance_domains() immediately. So the call graph
> > >> would be:
> > > 
> > > Oh right, since that only sets the softirq, this wouldn't be the issue,
> > > though we would need these changes if we were to incorporate any sort of
> > > nohz_kick_needed() logic into the nohz_idle_balance() code path correct?
> > 
> > I am sorry I don't quite get this. Can you please elaborate?
> 
> I think the scenario is that we are in nohz_idle_balance() and decide to
> bail out because we have pulled some tasks, but before leaving
> nohz_idle_balance() we want to check if more balancing is necessary
> using nohz_kick_needed() and potentially kick somebody to continue.

> Note that the balance cpu is currently skipped in nohz_idle_balance(),
> but if it wasn't the scenario would be possible.

This scenario would also be possible if we call rebalance_domains()
first again.

I'm wondering if adding the nohz_kick_needed(), ect... in
nohz_idle_balance() can address the 10 second latency issue while still
calling rebalance_domains() first, since it seems more ideal to try
balancing on the current awake CPU first, as you also have mentioned

> In that case, we can't rely on rq->idle_balance as it would not be
> up-to-date. Also, we may even want to use nohz_kick_needed(rq) where rq
> != this_rq, in which case we probably also want an updated status. It
> seems that rq->idle_balance is only updated at each tick.

Yup, that's about what I was describing.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/