linux-kernel - Re: [RFC PATCH 2/4] sched: Adding gang scheduling infrastrucure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87ehw02f8u.fsf@abhimanyu.in.ibm.com>
Date:	Tue, 20 Dec 2011 07:09:29 +0530
From:	Nikunj A Dadhania <nikunj@...ux.vnet.ibm.com>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	mingo@...e.hu, linux-kernel@...r.kernel.org,
	vatsa@...ux.vnet.ibm.com, bharata@...ux.vnet.ibm.com
Subject: Re: [RFC PATCH 2/4] sched: Adding gang scheduling infrastrucure

On Mon, 19 Dec 2011 16:51:44 +0100, Peter Zijlstra <peterz@...radead.org> wrote:
> On Mon, 2011-12-19 at 14:04 +0530, Nikunj A. Dadhania wrote:
> 
> > +	raw_spin_lock_irqsave(&rq->lock, flags);
> > +
> > +	/* Check if the runqueue has runnable tasks */
> > +	if (cfs_rq->nr_running) {
> > +		/* Favour this task group and set need_resched flag,
> > +		 * added by following patches */
> 
> That's just plain insanity, patch 3 is all of 4 lines, why split that
> and have an incomplete patch here?
> 
I will fold that in this patch.

> > +	}
> > +	raw_spin_unlock_irqrestore(&rq->lock, flags);
> > +}
> > +
> > +#define GANG_SCHED_GRANULARITY 8
> 
> Why have this magical number to begin with?
> 
We do not want to gang across the complete machine say 128cpus. Break it
to 16 independent gang. So that way we can scale up. 

This can be a sysctl or architecture specific define.

> > +void gang_sched(struct task_group *tg, struct rq *rq)
> > +{
> > +	/* We do not gang sched here */
> > +	if (rq->gang_leader == 0 || !tg || tg->gang == 0)
> > +		return;
> > +
> > +	/* Yes thats the leader */
> > +	if (rq->gang_leader == 1) {
> > +
> > +		if (!in_interrupt() && !irqs_disabled()) {
> 
> How can this ever happen, schedule() can't be called from interrupt
> context and post_schedule() ensures interrupts are enabled.
> 
Ah... thought that schedule can get called from interrupt
context. Sometime back I had some crash without this, let me remove this
and check it.

And smp_call_function_many required that, so those conditions. From the
function header;

 * You must not call this function with disabled interrupts or from a
 * hardware interrupt handler or from a bottom half handler. Preemption
 * must be disabled when calling this function.
 */

> > +			smp_call_function_many(rq->gang_cpumask,
> > +					gang_sched_member, tg, 0);
> 
> See this is just not going to happen..
> 
Why do you say that? I had trace functions in my debug code and I was
hitting gang_sched_member on the other cpus.

> > +
> > +		for_each_domain(cpu_of(rq), sd) {
> > +			count = 0;
> > +			for_each_cpu(i, sched_domain_span(sd))
> > +				count++;
> 
> That's just incompetent; there's cpumask_weight(), also that's called
> sd->span_weight.
> 
Let me go and check that out, will use them. It will definitely reduce
the code here.

Regards
Nikunj

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/