linux-kernel - Re: [patch] CFS scheduler, -v8

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <46386F2B.9050307@linux.vnet.ibm.com>
Date:	Wed, 02 May 2007 16:29:55 +0530
From:	Balbir Singh <balbir@...ux.vnet.ibm.com>
To:	Ingo Molnar <mingo@...e.hu>
CC:	linux-kernel@...r.kernel.org,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Con Kolivas <kernel@...ivas.org>,
	Nick Piggin <npiggin@...e.de>, Mike Galbraith <efault@....de>,
	Arjan van de Ven <arjan@...radead.org>,
	Peter Williams <pwil3058@...pond.net.au>,
	Thomas Gleixner <tglx@...utronix.de>, caglar@...dus.org.tr,
	Willy Tarreau <w@....eu>,
	Gene Heskett <gene.heskett@...il.com>, Mark Lord <lkml@....ca>,
	Zach Carter <linux@...hcarter.com>,
	buddabrod <buddabrod@...il.com>
Subject: Re: [patch] CFS scheduler, -v8

Ingo Molnar wrote:
> * Balbir Singh <balbir@...ux.vnet.ibm.com> wrote:
> 
>> With -v7 I would run the n/n+1 test. Basically on a system with n 
>> cpus, I would run n+1 tasks and see how their load is distributed. I 
>> usually find that the last two tasks would get stuck on one CPU on the 
>> system and would get half the cpu time as their other peers. I think 
>> this issue has been around for long even before CFS. But while I was 
>> investigating that, I found that with -v8, all the n+1 tasks are stuck 
>> on the same cpu.
> 
> i believe this problem is specific to powerpc - load is distributed fine 
> on i686/x86_64 and your sched_debug shows a cpu_load[0] == 0 on CPU#2 
> which is 'impossible'. (I sent a few suggestions off-Cc about how to 
> debug this.)
> 
> 	Ingo

Hi, Ingo

The suggestions helped, here is a fix tested on PowerPC only.

Patch and Description
=====================


Load balancing on PowerPC is broken. Running 5 tasks on a 4 cpu system
results in all 5 tasks running on the same CPU. Based on Ingo's feedback,
I instrumented and debugged update_load_fair().

The problem is with comparing a s64 values with (s64)ULONG_MAX, which
evaluates to -1. Then we check if exec_delta64 and fair_delta64 are greater
than (s64)ULONG_MAX (-1), if so we assign (s64)ULONG_MAX to the respective
values.

The fix is to compare these values against (s64)LONG_MAX and assign
(s64)LONG_MAX to exec_delta64 and fair_delta64 if they are greater than
(s64)LONG_MAX.

Tested on PowerPC, the regression is gone, tasks are load balanced as they
were in v7.

Output of top

  5614 root      20   0  4912  784  252 R   52  0.0   3:27.49 3 bash 

  5620 root      20   0  4912  784  252 R   47  0.0   3:07.38 2 bash 

  5617 root      20   0  4912  784  252 R   47  0.0   3:08.18 0 bash 

  5624 root      20   0  4912  784  252 R   26  0.0   1:42.97 1 bash 

  5621 root      20   0  4912  784  252 R   26  0.0   1:43.14 1 bash 


Tasks 5624 and 5621 getting half of their peer values is a separate issue
altogether.

Signed-off-by: Balbir Singh <balbir@...ux.vnet.ibm.com>
---

  kernel/sched.c |   10 +++++-----
  1 file changed, 5 insertions(+), 5 deletions(-)

diff -puN kernel/sched.c~cfs-fix-load-balancing-arith kernel/sched.c
--- linux-2.6.21/kernel/sched.c~cfs-fix-load-balancing-arith	2007-05-02 
16:16:20.000000000 +0530
+++ linux-2.6.21-balbir/kernel/sched.c	2007-05-02 16:16:47.000000000 +0530
@@ -1533,19 +1533,19 @@ static void update_load_fair(struct rq *
  	this_rq->prev_exec_clock = this_rq->exec_clock;
  	WARN_ON_ONCE(exec_delta64 <= 0);

-	if (fair_delta64 > (s64)ULONG_MAX)
-		fair_delta64 = (s64)ULONG_MAX;
+	if (fair_delta64 > (s64)LONG_MAX)
+		fair_delta64 = (s64)LONG_MAX;
  	fair_delta = (unsigned long)fair_delta64;

-	if (exec_delta64 > (s64)ULONG_MAX)
-		exec_delta64 = (s64)ULONG_MAX;
+	if (exec_delta64 > (s64)LONG_MAX)
+		exec_delta64 = (s64)LONG_MAX;
  	exec_delta = (unsigned long)exec_delta64;
  	if (exec_delta > TICK_NSEC)
  		exec_delta = TICK_NSEC;

  	idle_delta = TICK_NSEC - exec_delta;

-	tmp = SCHED_LOAD_SCALE * exec_delta / fair_delta;
+	tmp = (SCHED_LOAD_SCALE * exec_delta) / fair_delta;
  	tmp64 = (u64)tmp * (u64)exec_delta;
  	do_div(tmp64, TICK_NSEC);
  	this_load = (unsigned long)tmp64;
_


-- 
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/