[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAKfTPtDYXeCnYgR0yU2Ed-XBP9ezfk-j1gCgEyiReyQjD5hC6Q@mail.gmail.com>
Date: Fri, 5 Sep 2014 14:27:36 +0200
From: Vincent Guittot <vincent.guittot@...aro.org>
To: Preeti U Murthy <preeti@...ux.vnet.ibm.com>
Cc: "peterz@...radead.org" <peterz@...radead.org>,
Ingo Molnar <mingo@...nel.org>, Rik van Riel <riel@...hat.com>,
Morten Rasmussen <Morten.Rasmussen@....com>,
LKML <linux-kernel@...r.kernel.org>,
Mike Galbraith <efault@....de>,
Nicolas Pitre <nicolas.pitre@...aro.org>,
"daniel.lezcano@...aro.org" <daniel.lezcano@...aro.org>,
Dietmar Eggemann <dietmar.eggemann@....com>,
Kamalesh Babulal <kamalesh@...ux.vnet.ibm.com>,
Srikar Dronamraju <srikar@...ux.vnet.ibm.com>
Subject: Re: [QUERY] Confusing usage of rq->nr_running in load balancing
On 5 September 2014 14:19, Preeti U Murthy <preeti@...ux.vnet.ibm.com> wrote:
> Hi Vincent,
>
> On 09/03/2014 10:28 PM, Vincent Guittot wrote:
>> On 3 September 2014 14:21, Preeti U Murthy <preeti@...ux.vnet.ibm.com> wrote:
>>> Hi,
>>
>> Hi Preeti,
>>
>>>
>>> There are places in kernel/sched/fair.c in the load balancing part where
>>> rq->nr_running is used as against cfs_rq->nr_running. At least I could
>>> not make out why the former was used in the following scenarios.
>>> It looks to me that it can very well lead to incorrect load balancing.
>>> Also I did not pay attention to the numa balancing part of the code
>>> while skimming through this file to catch this scenario. There are a
>>> couple of places there too which need to be scrutinized.
>>>
>>> 1. load_balance(): The check (busiest->nr_running > 1)
>>> The load balancing would be futile if there are tasks of other
>>> scheduling classes, wouldn't it?
>>
>> agree with you
>>
>>>
>>> 2. active_load_balance_cpu_stop(): A similar check and a similar
>>> consequence as 1 here.
>>
>> agree with you
>>
>>>
>>> 3. nohz_kick_needed() : We check for more than one task on the runqueue
>>> and hence trigger load balancing even if there are rt-tasks.
>>
>> I can see one potentiel reason why rq->nr_running is interesting that
>> is the group capacity might have changed because of non cfs tasks
>> since last load balance. So we need to monitor the change of the
>> groups' capacity to ensure that the average load of each group is
>> still in the same level
>>
>>>
>>> 4. cpu_avg_load_per_task(): This stands out among the rest as an
>>> incorrect usage of rq->nr_running in place of cfs_rq->nr_running. We
>>> divide the load associated with the cfs_rq by the number of tasks on the
>>> rq. This will make the cfs_rq load look smaller.
>>
>> This one is solved in the consolidation of cpu_capacity patchset
>
> Sorry, but I don't see where in your patchset you have addressed this
> issue. Can you please point out the patch?
In [PATCH v5 03/12] sched: fix avg_load computation:
static unsigned long cpu_avg_load_per_task(int cpu)
{
struct rq *rq = cpu_rq(cpu);
- unsigned long nr_running = ACCESS_ONCE(rq->nr_running);
+ unsigned long nr_running = ACCESS_ONCE(rq->cfs.h_nr_running);
unsigned long load_avg = rq->cfs.runnable_load_avg;
Are you referring to another problem than the one above ?
Regards
Vincent
>
> Regards
> Preeti U Murthy
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists