Message-ID: <20150908071139.GI6455@byungchulpark-X58A-UD3R>
Date:	Tue, 8 Sep 2015 16:11:39 +0900
From:	Byungchul Park <byungchul.park@....com>
To:	Wanpeng Li <wanpeng.li@...mail.com>
Cc:	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...nel.org>, linux-kernel@...r.kernel.org,
	yuyang.du@...el.com
Subject: Re: [PATCH] sched: fix lose fair sleeper bonus in switch_to_fair()

On Tue, Sep 08, 2015 at 02:42:52PM +0800, Wanpeng Li wrote:
> On 9/8/15 2:32 PM, Byungchul Park wrote:
> >On Tue, Sep 08, 2015 at 03:14:26PM +0900, Byungchul Park wrote:
> >>On Tue, Sep 08, 2015 at 01:38:08PM +0800, Wanpeng Li wrote:
> >>>On 9/8/15 1:28 PM, Byungchul Park wrote:
> >>>>On Tue, Sep 08, 2015 at 11:46:01AM +0800, Wanpeng Li wrote:
> >>>>>On 9/7/15 10:02 PM, Peter Zijlstra wrote:
> >>>>>>Please always Cc at least the person who wrote the lines you modify.
> >>>>>>
> >>>>>>On Mon, Sep 07, 2015 at 05:45:20PM +0800, Wanpeng Li wrote:
> >>>>>>>The sleeper task will be normalized when moved from fair_sched_class, in
> >>>>>>>order that vruntime will be adjusted either the task is running or sleeping
> >>>>>>>when moved back. The nomalization in switch_to_fair for sleep task will
> >>>>>>>result in lose fair sleeper bonus in place_entity() once the vruntime -
> >>>>>>>cfs_rq->min_vruntime is big when moved from fair_sched_class.
> >>>>>>>
> >>>>>>>This patch fix it by adjusting vruntime just during migrating as original
> >>>>>>>codes since the vruntime of the task has usually NOT been normalized in
> >>>>>>>this case.
> >>>>>>Sorry, I cannot follow that at all. Maybe its me being sleep deprived,
> >>>>>>but could you try that again?
> >>>>>When changing away from the fair class while sleeping, relative
> >>>>>vruntime is calculated to handle the case sleep when moved from
> >>>>>fair_sched_class and running when moved to fair_sched_class. The
> >>>>i don't think relative vruntime is calculated to handle the special case
> >>>>you mentioned. i think the calculation is necessary for all cases detaching
> >>>Please refer why the relative vruntime caculation is introduced to
> >>>switched_from_fair(): https://lkml.org/lkml/2011/1/17/129
> >>hello,
> >>
> >>it is just a bug caused by not calculating a relative vruntime when
> >>detached a task from cfs_rq, which is necessary though.
> >>
> >>>>a task from a cfs_rq.
> >>>>
> >>>>>absolute vruntime will be calculated in enqueue_entity() either the
> >>>>>task is running or sleeping when moved back. The fair sleeper bonus
> >>>>i think absolute vruntime is calculated in enqueue_entuty() only when the
> >>>>task is on rq. therefore in the case that the task is not on rq,
> >>>>switched_to_fair() has to calculate the absolute vruntime instread.
> >>>Absolute vruntime is caculated in place_entity() which is called by
> >>>enqueue_entity() for DEQUEUE_SLEEP task.
> >>as you may know, place_entity() is not for calculating an absolute
> >>vruntime though.. anyway the important thing here is that, when a
> >>sleeping task is moved back to fair class, enqueue_entity() for
> >>DEQUEUE_SLEEP task won't be called.
> >you may talk about calling enqueue_entity() when the task is woken up,
> >not just when it is moved back. right?
> 
> Exactly.
> 
> >
> >even if yes, i think place_entity() should not be used directly for
> >calculating an absolute vruntime. it should be called after non/normalizing
> >operations.
> 
> The se->vruntime += cfs_rq->min_vruntime(in your switched_to_fair())
> which means that se->vruntime is bigger than cfs_rq->min_vruntime,

it is not always true since se->vruntime can have a negative value (even
though it is a unsigned type.. i think it can be another problem) by 
se->vruntime -= cfs_rq->min_vruntime in detach_task_cfs_rq().

> however, fair sleeper bonus is min_vuntime - sysctl_sched_latency/2,
> which means that max_vruntime() will select the absolute vruntime
> which is caculated in your switched_to_fair() as the se->vruntime,

since se->vruntime can have a negative value, max_vruntime() may select
the fair sleeper bonused value.

by the way, this logic is unchanged by my patch. which part of my patch
changed this kind of logic?

thanks,
byungchul

> then the fair sleeper bonus is lost in this case.
> 
> Regards,
> Wanpeng Li
> 
> >
> >>thanks,
> >>byungchul
> >>
> >>>Regards,
> >>>Wanpeng Li
> >>>
> >>>>>should be gained in place_entity() if the task is still sleeping.
> >>>>>However, after recent commit ( 23ec30ddd7c1306: 'sched: add two
> >>>>>functions for att(det)aching a task to(from) a cfs_rq'), the
> >>>>>absolute vruntime will be calculated in switched_to_fair(), so the
> >>>>>max_vruntime() which is called in place_entity() will select the
> >>>>>absolute vruntime which is calculated in switched_to_fair() as the
> >>>>>se->vruntime and lose the fair sleeper bonus.
> >>>>please refer my another reply, and let me know if i missed something.
> >>>>
> >>>>thanks,
> >>>>byungchul
> >>>>
> >>>>>Regards,
> >>>>>Wanpeng Li
> >>>>>
> >>>>>>>Signed-off-by: Wanpeng Li <wanpeng.li@...mail.com>
> >>>>>>>---
> >>>>>>>  kernel/sched/fair.c |   11 +++++++----
> >>>>>>>  1 files changed, 7 insertions(+), 4 deletions(-)
> >>>>>>>
> >>>>>>>diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> >>>>>>>index d26d3b7..eb9aa35 100644
> >>>>>>>--- a/kernel/sched/fair.c
> >>>>>>>+++ b/kernel/sched/fair.c
> >>>>>>>@@ -8005,9 +8005,6 @@ static void attach_task_cfs_rq(struct task_struct *p)
> >>>>>>>  	/* Synchronize task with its cfs_rq */
> >>>>>>>  	attach_entity_load_avg(cfs_rq, se);
> >>>>>>>-
> >>>>>>>-	if (!vruntime_normalized(p))
> >>>>>>>-		se->vruntime += cfs_rq->min_vruntime;
> >>>>>>>  }
> >>>>>>>  static void switched_from_fair(struct rq *rq, struct task_struct *p)
> >>>>>>>@@ -8066,14 +8063,20 @@ void init_cfs_rq(struct cfs_rq *cfs_rq)
> >>>>>>>  #ifdef CONFIG_FAIR_GROUP_SCHED
> >>>>>>>  static void task_move_group_fair(struct task_struct *p)
> >>>>>>>  {
> >>>>>>>+	struct sched_entity *se = &p->se;
> >>>>>>>+	struct cfs_rq *cfs_rq = cfs_rq_of(se);
> >>>>>>>+
> >>>>>>>  	detach_task_cfs_rq(p);
> >>>>>>>  	set_task_rq(p, task_cpu(p));
> >>>>>>>  #ifdef CONFIG_SMP
> >>>>>>>  	/* Tell se's cfs_rq has been changed -- migrated */
> >>>>>>>-	p->se.avg.last_update_time = 0;
> >>>>>>>+	se->avg.last_update_time = 0;
> >>>>>>>  #endif
> >>>>>>>  	attach_task_cfs_rq(p);
> >>>>>>>+
> >>>>>>>+	if (!vruntime_normalized(p))
> >>>>>>>+		se->vruntime += cfs_rq->min_vruntime;
> >>>>>>>  }
> >>>>>>>  void free_fair_sched_group(struct task_group *tg)
> >>>>>>>-- 
> >>>>>>>1.7.1
> >>>>>>>
> >>>>>--
> >>>>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> >>>>>the body of a message to majordomo@...r.kernel.org
> >>>>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>>>>Please read the FAQ at  http://www.tux.org/lkml/
> >>>--
> >>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> >>>the body of a message to majordomo@...r.kernel.org
> >>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>>Please read the FAQ at  http://www.tux.org/lkml/
> >>--
> >>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> >>the body of a message to majordomo@...r.kernel.org
> >>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>Please read the FAQ at  http://www.tux.org/lkml/
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists