[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <5D3A1D31-B64C-4B4E-AE62-41A85C8E42D0@tencent.com>
Date: Tue, 21 Jul 2020 07:35:09 +0000
From: benbjiang(蒋彪)
<benbjiang@...cent.com>
To: Vineeth Remanan Pillai <vpillai@...italocean.com>,
Peter Zijlstra <peterz@...radead.org>
CC: Nishanth Aravamudan <naravamudan@...italocean.com>,
Julien Desfossez <jdesfossez@...italocean.com>,
Peter Zijlstra <peterz@...radead.org>,
"Tim Chen" <tim.c.chen@...ux.intel.com>,
"mingo@...nel.org" <mingo@...nel.org>,
"tglx@...utronix.de" <tglx@...utronix.de>,
"pjt@...gle.com" <pjt@...gle.com>,
"torvalds@...ux-foundation.org" <torvalds@...ux-foundation.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"subhra.mazumdar@...cle.com" <subhra.mazumdar@...cle.com>,
"fweisbec@...il.com" <fweisbec@...il.com>,
"keescook@...omium.org" <keescook@...omium.org>,
"kerrnel@...gle.com" <kerrnel@...gle.com>,
Phil Auld <pauld@...hat.com>, Aaron Lu <aaron.lwe@...il.com>,
Aubrey Li <aubrey.intel@...il.com>,
Valentin Schneider <valentin.schneider@....com>,
Mel Gorman <mgorman@...hsingularity.net>,
Pawan Gupta <pawan.kumar.gupta@...ux.intel.com>,
Paolo Bonzini <pbonzini@...hat.com>,
Joel Fernandes <joelaf@...gle.com>,
"Joel Fernandes (Google)" <joel@...lfernandes.org>,
"vineethrp@...il.com" <vineethrp@...il.com>,
"Chen Yu" <yu.c.chen@...el.com>,
Christian Brauner <christian.brauner@...ntu.com>
Subject: Re: [RFC PATCH 07/16] sched/fair: Fix forced idle sibling starvation
corner case(Internet mail)
Hi,
> On Jul 1, 2020, at 5:32 AM, Vineeth Remanan Pillai <vpillai@...italocean.com> wrote:
>
> From: vpillai <vpillai@...italocean.com>
>
> If there is only one long running local task and the sibling is
> forced idle, it might not get a chance to run until a schedule
> event happens on any cpu in the core.
>
> So we check for this condition during a tick to see if a sibling
> is starved and then give it a chance to schedule.
>
> Signed-off-by: Vineeth Remanan Pillai <vpillai@...italocean.com>
> Signed-off-by: Julien Desfossez <jdesfossez@...italocean.com>
> ---
> kernel/sched/fair.c | 39 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 39 insertions(+)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index ae17507533a0..49fb93296e35 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -10613,6 +10613,40 @@ static void rq_offline_fair(struct rq *rq)
>
> #endif /* CONFIG_SMP */
>
> +#ifdef CONFIG_SCHED_CORE
> +static inline bool
> +__entity_slice_used(struct sched_entity *se)
> +{
> + return (se->sum_exec_runtime - se->prev_sum_exec_runtime) >
> + sched_slice(cfs_rq_of(se), se);
> +}
> +
> +/*
> + * If runqueue has only one task which used up its slice and if the sibling
> + * is forced idle, then trigger schedule to give forced idle task a chance.
> + */
> +static void resched_forceidle_sibling(struct rq *rq, struct sched_entity *se)
> +{
> + int cpu = cpu_of(rq), sibling_cpu;
> +
> + if (rq->cfs.nr_running > 1 || !__entity_slice_used(se))
> + return;
> +
> + for_each_cpu(sibling_cpu, cpu_smt_mask(cpu)) {
> + struct rq *sibling_rq;
> + if (sibling_cpu == cpu)
> + continue;
> + if (cpu_is_offline(sibling_cpu))
> + continue;
> +
> + sibling_rq = cpu_rq(sibling_cpu);
> + if (sibling_rq->core_forceidle) {
> + resched_curr(sibling_rq);
> + }
> + }
> +}
> +#endif
> +
> /*
> * scheduler tick hitting a task of our scheduling class.
> *
> @@ -10636,6 +10670,11 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
>
> update_misfit_status(curr, rq);
> update_overutilized_status(task_rq(curr));
> +
> +#ifdef CONFIG_SCHED_CORE
> + if (sched_core_enabled(rq))
> + resched_forceidle_sibling(rq, &curr->se);
> +#endif
Hi,
resched_forceidle_sibling depends on tick, but there could be no tick in 1s(scheduler_tick_max_derferment) after
entering nohz_full mode.
And when enable nohz_full, cpu will enter nohz_full mode frequently when *there is only one long running local task*.
That means the siblings rescheduling would be delayed much more than sched_slice(), could be unfair and result in
big latency.
Should we restrict cpu with forced-idle sibling entering nohz_full mode by adding specific flag and checking it before
stop tick?
Or we can do rescheduling on siblings in task_tick_idle by checking starvation time? :)
Thx
Regard,
Jiang
> }
>
> /*
> --
> 2.17.1
>
>
Powered by blists - more mailing lists