linux-kernel - Re: CFS Bandwidth Control - Test results of cgroups tasks pinned vs unpinnede

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1316526969.13664.31.camel@twins>
Date:	Tue, 20 Sep 2011 15:56:09 +0200
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	Kamalesh Babulal <kamalesh@...ux.vnet.ibm.com>
Cc:	Srivatsa Vaddagiri <vatsa@...ux.vnet.ibm.com>,
	Paul Turner <pjt@...gle.com>,
	Vladimir Davydov <vdavydov@...allels.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	Bharata B Rao <bharata@...ux.vnet.ibm.com>,
	Dhaval Giani <dhaval.giani@...il.com>,
	Vaidyanathan Srinivasan <svaidy@...ux.vnet.ibm.com>,
	Ingo Molnar <mingo@...e.hu>,
	Pavel Emelianov <xemul@...allels.com>
Subject: Re: CFS Bandwidth Control - Test results of cgroups tasks pinned vs
 unpinnede

On Mon, 2011-09-19 at 23:21 +0530, Kamalesh Babulal wrote:
> * Peter Zijlstra <a.p.zijlstra@...llo.nl> [2011-09-15 23:48:43]:
> 
> > On Thu, 2011-09-15 at 23:25 +0530, Kamalesh Babulal wrote:
> > > 2.6.38          | 1551  | 48    | 62    | 47    | 50    |
> > > ----------------+-------+-------+-------+-------+-------+
> > > 2.6.39          | 3784  | 457   | 722   | 3209  | 1037  | 
> > 
> > I'd say we wrecked it going from .38 to .39 and only made it worse after
> > that.
> 
> after reverting the commit 866ab43efd325fae8889ea, of the patches 
> went between .38 and .39 reduces the ping pong of the tasks.
> 
> ------------------------+-------+-------+-------+-------+-------+
> Kernel			| Run 1	| Run 2	| Run 3	| Run 4	| Run 5	|
> ------------------------+-------+-------+-------+-------+-------+
> 2.6.39	        	| 1542  | 2172  | 2727  | 120   | 3681  |
> ------------------------+-------+-------+-------+-------+-------+
> 2.6.39 (with    	|       |       |       |       |       |
> 866ab43efd reverted)	| 65	| 78	| 58	| 99 	| 62	|
> ------------------------+-------+-------+-------+-------+-------+
> 3.1-rc4+tip		|	|	|	|	|	|
> (e467f18f945c)		| 1219	| 2037	| 1943	| 772	| 1701	|
> ------------------------+-------+-------+-------+-------+-------+
> 3.1-rc4+tip (e467f18f9)	|	|	|	|	|	|
> (866ab43efd reverted)	| 64	| 45	| 59	| 59	| 69	|
> ------------------------+-------+-------+-------+-------+-------+

Right, so reverting that breaks the cpuset/cpuaffinity thing again :-(

Now I'm not quite sure why group_imb gets toggled in this use-case at
all, having put a trace_printk() in, we get:

           <...>-1894  [006]   704.056250: find_busiest_group: max: 2048, min: 0, avg: 1024, nr: 2
     kworker/1:1-101   [001]   706.305523: find_busiest_group: max: 3072, min: 0, avg: 1024, nr: 3

Which is of course a bad state to be in, but we also get:

    migration/17-73    [017]   706.284191: find_busiest_group: max: 1024, min: 0, avg: 512, nr: 2
          <idle>-0     [003]   706.325435: find_busiest_group: max: 1250, min: 440, avg: 1024, nr: 2

on a CGROUP=n kernel.. which I think we can attribute to races.

When I enable tracing I also get some good runs, so it smells like the
lb does one bad thing and instead of correcting it it makes it worse.

It looks like its set-off by a mass-wakeup of random crap that really
shouldn't be waking at all, I mean who needs automount to wakeup, or
whatever the fuck rtkit-daemon is. I'm pretty sure my bash loops don't
do anything remotely related to those.

Anyway, once enough random crap wakes up, the load-balancer goes shift
stuff around, once we hit the group_imb conditions we seem to get stuck
in a bad state instead of getting out of it.

Bah!






--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/