linux-kernel - [BUG] sched: fair group's bug

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 08 Jan 2009 09:30:35 +0800
From:	Miao Xie <miaox@...fujitsu.com>
To:	Ingo Molnar <mingo@...e.hu>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>
CC:	Linux-Kernel <linux-kernel@...r.kernel.org>
Subject: [BUG] sched: fair group's bug

I tested fair group scheduler on my hyper-threading x86_64 box(2 CPU * 2 HT)
and found the deviation of the groups' CPU usage was larger than 2.6.26
when *offline* a CPU or do hotplug frequently. It is less than 1% On 2.6.26，but
On current kernel, it is often greater than 4%, even than 10% by accident.

A test program which reproduces the problem on current kernel is attached.
This program forks a lot of child tasks and attachs these child tasks into each
CPU controller group, then the parent task gets and checks the CPU usage of
every group every 5 seconds.(All of the child tasks do the same work - repeat
doing sqrt)
						Child Task 1
						  while(!end)
						      sqrt(f);
				Group 1		......
						Child Task m
						  while(!end)
						      sqrt(f);
				-----------------------------
Parent Task             	Group 2		......
  get and check the CPU		......		......
  usage of every group		-----------------------------
						Child Task m*n-m+1
						  while(!end)
						      sqrt(f);
				Group n		......
						Child Task m*n
						  while(!end)
						      sqrt(f);

Steps to reproduce:
# mkdir /dev/cpuctl
# mount -t cgroup -o cpu,noprefix xxx /dev/cpuctl
# ./cpuctl

output on current kernel:
-------------------------
Group   Shares  Actual(%)       Expect(%)
0       1024    41.68           50.00
1       1024    58.32           50.00
0th group's usage is out of range(deviation = 8.32)
1th group's usage is out of range(deviation = 8.32)
--------------------------

output on 2.6.26
-------------------------
Group   Shares  Actual(%)       Expect(%)
0       1024    50.03           50.00
1       1024    49.97           50.00
--------------------------

On 2.6.26, the deviation may be greater than 4% at the beginning, but the scheduler
adjusts soon, and don't occur the large deviation for ever.

Bisect located below patch:
commit c09595f63bb1909c5dc4dca288f4fe818561b5f3
Author: Peter Zijlstra <a.p.zijlstra@...llo.nl>
Date:   Fri Jun 27 13:41:14 2008 +0200

    sched: revert revert of: fair-group: SMP-nice for group scheduling

    Try again..

    Initial commit: 18d95a2832c1392a2d63227a7a6d433cb9f2037e
    Revert: 6363ca57c76b7b83639ca8c83fc285fa26a7880e

    Signed-off-by: Peter Zijlstra <a.p.zijlstra@...llo.nl>
    Cc: Srivatsa Vaddagiri <vatsa@...ux.vnet.ibm.com>
    Cc: Mike Galbraith <efault@....de>
    Signed-off-by: Ingo Molnar <mingo@...e.hu>


Besides that, We found other problems by the attached program.
1. some tasks are hungry in the fair group.
	Steps to reproduce:
	# mkdir /dev/cpuctl
	# mount -t cgroup -o cpu,noprefix xxx /dev/cpuctl
	# ./cpuctl -g 1 -v
	--------------------
	1th Check Result:
	Group	Shares	Actual(%)	Expect(%)
	0	1024	100.00		100.00
	Each task's usage:
		Task in Group 0:
			Task	Usage(%)
			5395	0.000000
			5396	0.000000
		 	5397	0.000000
			5398	16.677785
			5399	16.677785
			5400	16.744496
			5401	16.611074
			5402	33.288859

2. some groups broke the limit of the fair group and get more CPU time When
   the groups is hiberarchy. Such as:
		top group
		    |
		 group 1
		/	\
	      task1	group 2
			   |
			task 2
	Steps to reproduce:
	# mkdir /dev/cpuctl
	# mount -t cgroup -o cpu,noprefix xxx /dev/cpuctl
	# ./cpuctl -H
	-------------------------
	Group	Shares	Actual(%)	Expect(%)
	0	1024	60.17		88.89
	1	1024    39.83		11.11

View attachment "cpuctl.c" of type "text/x-csrc" (18160 bytes)