linux-kernel - Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20151103084501.289ec5d1@yairi>
Date:	Tue, 3 Nov 2015 08:45:01 -0800
From:	Jacob Pan <jacob.jun.pan@...ux.intel.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	Thomas Gleixner <tglx@...utronix.de>,
	LKML <linux-kernel@...r.kernel.org>,
	Arjan van de Ven <arjan@...ux.intel.com>,
	Paul Turner <pjt@...gle.com>, Len Brown <len.brown@...el.com>,
	Srinivas Pandruvada <srinivas.pandruvada@...ux.intel.com>,
	Tim Chen <tim.c.chen@...ux.intel.com>,
	Andi Kleen <andi.kleen@...el.com>,
	Rafael Wysocki <rafael.j.wysocki@...el.com>
Subject: Re: [RFC PATCH 3/3] sched: introduce synchronized idle injection

On Tue, 3 Nov 2015 14:31:20 +0100
Peter Zijlstra <peterz@...radead.org> wrote:

> > @@ -5136,6 +5148,16 @@ pick_next_task_fair(struct rq *rq, struct
> > task_struct *prev) struct task_struct *p;
> >  	int new_tasks;
> >  
> > +#ifdef CONFIG_CFS_IDLE_INJECT
> > +	if (cfs_rq->force_throttled &&
> > +		!idle_cpu(cpu_of(rq)) &&
> > +		!unlikely(local_softirq_pending())) {
> > +		/* forced idle, pick no task */
> > +		trace_sched_cfs_idle_inject(cpu_of(rq), 1);
> > +		update_curr(cfs_rq);
> > +		return NULL;
> > +	}
> > +#endif
> >  again:
> >  #ifdef CONFIG_FAIR_GROUP_SCHED
> >  	if (!cfs_rq->nr_running)
> 
> So this is horrible...
> 
> This is a fast path, and you just put at least one cachemiss in it, a
> branch (without hint) and some goofy code (wth are we checking
> softirqs?).
> 
softirq is checked here since it is one of the conditions to stop
sched tick. can_stop_idle_tick(). but we don't have to check here, you
are right.
> How about you frob things such that cfs_rq->nr_running == 0 and we'll
> hit the idle: path, at that point you can test if we're forced idle
> and skip the load-balancing attempt.
> 
> There's probably a fair number of icky cases to deal with if you frob
> cfs_rq->nr_running, like the enqueue path which will add to it. We'll
> have to come up with something to not slow that down either.
> 
> The thing is, both schedule and enqueue are very hot and this is code
> that will 'never' run.
Fair enough, I will give that a try, I guess it would be costly
and hard to scale if we were to dequeue/enqueue every se for every
period of injection plus locking. Let me get some data first.

I understand we don't want to sacrifice the hot patch for some code
almost 'never' run. But I also have follow up plan to use this code for
consolidating/synchronizing idle during balanced semi-active workload.
In that case, it may run more often. e.g.

Before:
CPU0 ______||| ||  |___________| || || |_____
CPU1 _________||| ||  |_______| || |_______

After:

CPU0 ______||| ||  |___________| || || |_____
CPU1 ______||| ||  |___________| || |_______

The goal is to have overlapping idle time if the load is already
balanced. The energy saving can be significant.

Jacob
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/