linux-kernel - Re: [CFS Bandwidth Control v4 3/7] sched: throttle cfs_rq entities which exceed their local quota

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 25 Feb 2011 19:28:56 +0530
From:	Bharata B Rao <bharata@...ux.vnet.ibm.com>
To:	Paul Turner <pjt@...gle.com>
Cc:	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	linux-kernel@...r.kernel.org,
	Dhaval Giani <dhaval@...ux.vnet.ibm.com>,
	Balbir Singh <balbir@...ux.vnet.ibm.com>,
	Vaidyanathan Srinivasan <svaidy@...ux.vnet.ibm.com>,
	Gautham R Shenoy <ego@...ibm.com>,
	Srivatsa Vaddagiri <vatsa@...ibm.com>,
	Kamalesh Babulal <kamalesh@...ux.vnet.ibm.com>,
	Ingo Molnar <mingo@...e.hu>,
	Pavel Emelyanov <xemul@...nvz.org>,
	Herbert Poetzl <herbert@...hfloor.at>,
	Avi Kivity <avi@...hat.com>,
	Chris Friesen <cfriesen@...tel.com>,
	Nikhil Rao <ncrao@...gle.com>
Subject: Re: [CFS Bandwidth Control v4 3/7] sched: throttle cfs_rq entities
 which exceed their local quota

On Thu, Feb 24, 2011 at 07:10:58PM -0800, Paul Turner wrote:
> On Wed, Feb 23, 2011 at 5:32 AM, Peter Zijlstra <a.p.zijlstra@...llo.nl> wrote:
> > On Tue, 2011-02-15 at 19:18 -0800, Paul Turner wrote:
> 
> >> +     update_cfs_load(cfs_rq, 0);
> >> +
> >> +     /* prevent previous buddy nominations from re-picking this se */
> >> +     clear_buddies(cfs_rq_of(se), se);
> >> +
> >> +     /*
> >> +      * It's possible for the current task to block and re-wake before task
> >> +      * switch, leading to a throttle within enqueue_task->update_curr()
> >> +      * versus an an entity that has not technically been enqueued yet.
> >
> > I'm not quite seeing how this would happen.. care to expand on this?
> >
> 
> I'm not sure the example Bharata gave is correct -- I'm going to treat
> that discussion separately as it's not the intent here.

Just for the record, my examples were not given for the above question from
Peter.

I answered two questions and I am tempted to stand by those until proven
wrong :)

1. Why do we have cfs_rq_throtted() check in dequeue_task_fair() ? 
( => How could we be running if our parent was throttled ?)

Consider the following hierarchy.

Root Group
   |
   |
Group 1 (Bandwidth constrained group)
   |
   |
Group 2 (Infinite runtime group)

Assume both the groups have tasks in them.

When Group 1 is throttled, its cfs_rq is marked throttled, and is removed from
Root group's runqueue. But leaf tasks in Group 2 continue to be enqueued in
Group 1's runqueue.

Load balancer kicks in on CPU A and figures out that it can pull a few tasks
from CPU B (busiest_cpu). It iterates through all the task groups
(load_balance_fair) and considers Group 2 also. It tries to pull a task from
CPU B's cfs_rq for Group 2. I don't see anything that would prevent the
load balancer from bailing out here. Note that Group 2 is technically
not throttled, only its parent Group 1 is. Load balancer goes ahead and
starts pulling individual tasks from Group 2's cfs_rq on CPU B. This
results in dequeuing of task whose hierarchy is throttled.

When load balancer iterates through Group 1's cfs_rqs, the situation is
different because we have already marked Group 1's cfs_rqs as throttled.
And we check this in load_balance_fair() and bail out from pulling tasks
from throttled hierarchy.

This is my understanding. Let me know what I miss. Specifically I would
like to understand how do you ensure that load balancer doesn't consider
tasks from throttled cfs_rqs for pulling.

2. Why there is cfs_rq_throttled() check in account_cfs_rq_quota() ?

In addition to the case you described, I believe the situation I described
is also valid.

Regards,
Bharata.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/