linux-kernel - Re: [PATCH 6/6] sched, fair: Rework and comment the group

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130816103438.GX3008@twins.programming.kicks-ass.net>
Date:	Fri, 16 Aug 2013 12:34:38 +0200
From:	Peter Zijlstra <peterz@...radead.org>
To:	Ingo Molnar <mingo@...nel.org>,
	Joonsoo Kim <iamjoonsoo.kim@....com>
Cc:	linux-kernel@...r.kernel.org, Mike Galbraith <efault@....de>,
	Paul Turner <pjt@...gle.com>, Alex Shi <alex.shi@...el.com>,
	Preeti U Murthy <preeti@...ux.vnet.ibm.com>,
	Vincent Guittot <vincent.guittot@...aro.org>,
	Morten Rasmussen <morten.rasmussen@....com>,
	Namhyung Kim <namhyung@...nel.org>,
	Joonsoo Kim <js1304@...il.com>, Rik van Riel <riel@...riel.com>
Subject: Re: [PATCH 6/6] sched, fair: Rework and comment the group_imb code

On Fri, Aug 16, 2013 at 12:12:27PM +0200, Peter Zijlstra wrote:
> +/*
> + * Group imbalance indicates (and tries to solve) the problem where balancing
> + * groups is inadequate due to tsk_cpus_allowed() constraints.
> + *
> + * Imagine a situation of two groups of 4 cpus each and 4 tasks each with a
> + * cpumask covering 1 cpu of the first group and 3 cpus of the second group.
> + * Something like:
> + *
> + * 	{ 0 1 2 3 } { 4 5 6 7 }
> + * 	        *     * * *

And yes, people think this is a reasonable thing to do :/

> + *
> + * If we were to balance group-wise we'd place two tasks in the first group and
> + * two tasks in the second group. Clearly this is undesired as it will overload
> + * cpu 3 and leave one of the cpus in the second group unused.
> + *
> + * The current solution to this issue is detecting the skew in the first group
> + * by noticing it has a cpu that is overloaded while the remaining cpus are
> + * idle -- or rather, there's a distinct imbalance in the cpus; see
> + * sg_imbalanced().
> + *
> + * When this is so detected; this group becomes a candidate for busiest; see
> + * update_sd_pick_busiest(). And calculcate_imbalance() and
> + * find_busiest_group() avoid some of the usual balance conditional to allow it
> + * to create an effective group imbalance.
> + *
> + * This is a somewhat tricky proposition since the next run might not find the
> + * group imbalance and decide the groups need to be balanced again. A most
> + * subtle and fragile situation.
> + */

One of the things that I know about which can (and does) go wrong with
this is that we typically pull 'all' tasks to the balance cpu only to
then let future (lower) load-balance passes spread that out again.

But until its spread out, we have this spike in the task distribution
that will trigger the group_imb condition.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/