linux-kernel - Re: [PATCH v4 01/12] sched: fix imbalance flag reset

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAKfTPtBS6Ri-hNf+M_BGDxhpXEwWP=n0zmTi5-pdxGi0dMF6eQ@mail.gmail.com>
Date:	Tue, 25 Nov 2014 10:04:58 +0100
From:	Vincent Guittot <vincent.guittot@...aro.org>
To:	Wanpeng Li <kernellwp@...il.com>
Cc:	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...nel.org>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	Preeti U Murthy <preeti@...ux.vnet.ibm.com>,
	Russell King - ARM Linux <linux@....linux.org.uk>,
	LAK <linux-arm-kernel@...ts.infradead.org>,
	Rik van Riel <riel@...hat.com>,
	Morten Rasmussen <Morten.Rasmussen@....com>,
	Mike Galbraith <efault@....de>,
	Nicolas Pitre <nicolas.pitre@...aro.org>,
	"linaro-kernel@...ts.linaro.org" <linaro-kernel@...ts.linaro.org>,
	Daniel Lezcano <daniel.lezcano@...aro.org>,
	Dietmar Eggemann <dietmar.eggemann@....com>
Subject: Re: [PATCH v4 01/12] sched: fix imbalance flag reset

On 25 November 2014 at 00:47, Wanpeng Li <kernellwp@...il.com> wrote:
> Hi Vincent,
> On 7/29/14, 1:51 AM, Vincent Guittot wrote:
>>
>> The imbalance flag can stay set whereas there is no imbalance.
>>
>> Let assume that we have 3 tasks that run on a dual cores /dual cluster
>> system.
>> We will have some idle load balance which are triggered during tick.
>> Unfortunately, the tick is also used to queue background work so we can
>> reach
>> the situation where short work has been queued on a CPU which already runs
>> a
>> task. The load balance will detect this imbalance (2 tasks on 1 CPU and an
>> idle
>> CPU) and will try to pull the waiting task on the idle CPU. The waiting
>> task is
>> a worker thread that is pinned on a CPU so an imbalance due to pinned task
>> is
>> detected and the imbalance flag is set.
>> Then, we will not be able to clear the flag because we have at most 1 task
>> on
>> each CPU but the imbalance flag will trig to useless active load balance
>> between the idle CPU and the busy CPU.
>>
>> We need to reset of the imbalance flag as soon as we have reached a
>> balanced
>> state. If all tasks are pinned, we don't consider that as a balanced state
>> and
>> let the imbalance flag set.
>>
>> Reviewed-by: Preeti U Murthy <preeti@...ux.vnet.ibm.com>
>> Signed-off-by: Vincent Guittot <vincent.guittot@...aro.org>
>> ---
>>   kernel/sched/fair.c | 23 +++++++++++++++++++----
>>   1 file changed, 19 insertions(+), 4 deletions(-)
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index 923fe32..7eb9126 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -6672,10 +6672,8 @@ static int load_balance(int this_cpu, struct rq
>> *this_rq,
>>                 if (sd_parent) {
>>                         int *group_imbalance =
>> &sd_parent->groups->sgc->imbalance;
>>   -                     if ((env.flags & LBF_SOME_PINNED) && env.imbalance
>> > 0) {
>> +                       if ((env.flags & LBF_SOME_PINNED) && env.imbalance
>> > 0)
>>                                 *group_imbalance = 1;
>> -                       } else if (*group_imbalance)
>> -                               *group_imbalance = 0;
>
>
> As you mentioned above " We need to reset of the imbalance flag as soon as
> we have reached a balanced state. " I think the codes before your patch have
> already do this, where I miss? Great thanks for your patient. ;-)

The previous code was called only when busiest->nr_running > 1.  The
background activity will be on the rq only 1 tick per few seconds and
we will set qroup_imbalance when the background activity is on the rq.
Then, during the next load balances, the qroup_imbalance is still set
but we can't clear qroup_imbalance  because we have only 1 task per rq

Regards,
Vincent

>
> Regards,
> Wanpeng Li
>
>
>>                 }
>>                 /* All tasks on this runqueue were pinned by CPU affinity
>> */
>> @@ -6686,7 +6684,7 @@ static int load_balance(int this_cpu, struct rq
>> *this_rq,
>>                                 env.loop_break = sched_nr_migrate_break;
>>                                 goto redo;
>>                         }
>> -                       goto out_balanced;
>> +                       goto out_all_pinned;
>>                 }
>>         }
>>   @@ -6760,6 +6758,23 @@ static int load_balance(int this_cpu, struct rq
>> *this_rq,
>>         goto out;
>>     out_balanced:
>> +       /*
>> +        * We reach balance although we may have faced some affinity
>> +        * constraints. Clear the imbalance flag if it was set.
>> +        */
>> +       if (sd_parent) {
>> +               int *group_imbalance = &sd_parent->groups->sgc->imbalance;
>> +
>> +               if (*group_imbalance)
>> +                       *group_imbalance = 0;
>> +       }
>> +
>> +out_all_pinned:
>> +       /*
>> +        * We reach balance because all tasks are pinned at this level so
>> +        * we can't migrate them. Let the imbalance flag set so parent
>> level
>> +        * can try to migrate them.
>> +        */
>>         schedstat_inc(sd, lb_balanced[idle]);
>>         sd->nr_balance_failed = 0;
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/