linux-kernel - Re: CFS Bandwidth Control - Test results of cgroups tasks pinned vs unpinned

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4DF73514.4080901@jp.fujitsu.com>
Date:	Tue, 14 Jun 2011 19:16:52 +0900
From:	Hidetoshi Seto <seto.hidetoshi@...fujitsu.com>
To:	Kamalesh Babulal <kamalesh@...ux.vnet.ibm.com>
CC:	Paul Turner <pjt@...gle.com>, linux-kernel@...r.kernel.org,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Bharata B Rao <bharata@...ux.vnet.ibm.com>,
	Dhaval Giani <dhaval.giani@...il.com>,
	Balbir Singh <balbir@...ux.vnet.ibm.com>,
	Vaidyanathan Srinivasan <svaidy@...ux.vnet.ibm.com>,
	Srivatsa Vaddagiri <vatsa@...ibm.com>,
	Ingo Molnar <mingo@...e.hu>, Pavel Emelyanov <xemul@...nvz.org>
Subject: Re: CFS Bandwidth Control - Test results of cgroups tasks pinned
 vs unpinned

(2011/06/08 0:45), Kamalesh Babulal wrote:
> Hi All,
> 
>     In our test environment, while testing the CFS Bandwidth V6 patch set 
> on top of 55922c9d1b84. We observed that the CPU's idle time is seen
> between 30% to 40% while running CPU bound test, with the cgroups tasks 
> not pinned to the CPU's. Whereas in the inverse case, where the cgroups 
> tasks are pinned to the CPU's, the idle time seen is nearly zero.

I've some test with your test script but I'm not sure whether it is really
a considerable problem. Am I missing the point?

I add -c option to your script to toggle pinning (1:pinned, 0:not pinned).
In short the results in my environment (16 cpu, 4 quad core) are:

				# group's usage
 -b 0 -p 0 -c 0 : Idle = 0%	(12,12,25,25,25)
 -b 0 -p 0 -c 1 : Idle = 0%	(6,6,12,25,50)
 -b 0 -p 1 -c * : Idle = 0%	(6,6,12,25,50)
 -b 1 -p 0 -c 0 : Idle = ~25%	(6,6,12,25,25)
 -b 1 -p 0 -c 1 : Idle = 0%	(6,6,12,25,50)
 -b 1 -p 1 -c * : Idle = 0%	(6,6,12,25,50)	

In my understanding is correct, when -p0, there are 5 groups (with share=1024)
and each group has 2,2,4,8,16 subgroups, so a subgroup in /1 is weighted 8 times
higher than one in /5.  And when -p1, share of 5 parent groups are promoted and
all subgroups are evenly weighted.
With -p0 the cpu usage of 5 groups is going to be 20,20,20,20,20 but group /1
and /2 have only 2 subgroups for each, so even if /1 and /2 fully use 2 cpus
for each the usage will be 12,12,25,25,25.

OTOH the bandwidth of a subgroup is 250000/500000 (=0.5 cpu), so in case of
Idle=0% the cpu usage of groups are likely be 6,6,12,25,50%. 

The question is what happen if both are mixed.

For example in case of your unpinned Idle=34.8%: 

> Average CPU Idle percentage 34.8% (as explained above in the Idle time measured)
> Bandwidth shared with remaining non-Idle 65.2%

> Bandwidth of Group 1 = 9.2500 i.e = 6.0300% of non-Idle CPU time 65.2%
> Bandwidth of Group 2 = 9.0400 i.e = 5.8900% of non-Idle CPU time 65.2%
> Bandwidth of Group 3 = 16.9300 i.e = 11.0300% of non-Idle CPU time 65.2%
> Bandwidth of Group 4 = 27.9300 i.e = 18.2100% of non-Idle CPU time 65.2%
> Bandwidth of Group 5 = 36.8300 i.e = 24.0100% of non-Idle CPU time 65.2%

The usage is 6,6,11,18,24.
It looks like that group /1 to /3 are limited by bandwidth, while group /5 is
limited by share. (I have no idea about the noise on /4 here)

BTW since pinning in your script always pin a couple of subgroup in a same
group to a cpu, subgroups are weighted evenly everywhere so as the result
share doesn't work for these cases.

Thanks,
H.Seto

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/