linux-kernel - Re: [PATCH 2/3] sched: Unloop sched avg decaying

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 30 Jun 2016 15:20:37 +0200
From:	Peter Zijlstra <peterz@...radead.org>
To:	Frederic Weisbecker <fweisbec@...il.com>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Ingo Molnar <mingo@...nel.org>, Mike Galbraith <efault@....de>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH 2/3] sched: Unloop sched avg decaying

On Thu, Jun 30, 2016 at 02:52:26PM +0200, Frederic Weisbecker wrote:
> On Tue, Jun 14, 2016 at 05:58:42PM +0200, Peter Zijlstra wrote:

> > Why not add the division to the nohz exit path only?
> 
> It would be worse I think because we may exit much more often from nohz
> than we reach a sched_avg_period().
> 
> So the only safe optimization I can do for now is:

How about something like this then?

---

 kernel/sched/core.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 3387e4f14fc9..fd1ae4c4105f 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -665,9 +665,23 @@ bool sched_can_stop_tick(struct rq *rq)
 
 void sched_avg_update(struct rq *rq)
 {
-	s64 period = sched_avg_period();
+	s64 delta, period = sched_avg_period();
 
-	while ((s64)(rq_clock(rq) - rq->age_stamp) > period) {
+	delta = (s64)(rq_clock(rq) - rq->age_stamp);
+	if (likely(delta < period))
+		return;
+
+	if (unlikely(delta > 3*period)) {
+		int pending;
+		u64 rem;
+
+		pending = div64_u64_rem(delta, period, &rem);
+		rq->age_stamp += delta - rem;
+		rq->rt_avg >>= pending;
+		return;
+	}
+
+	while (delta > period) {
 		/*
 		 * Inline assembly required to prevent the compiler
 		 * optimising this loop into a divmod call.
@@ -675,6 +689,7 @@ void sched_avg_update(struct rq *rq)
 		 */
 		asm("" : "+rm" (rq->age_stamp));
 		rq->age_stamp += period;
+		delta -= period;
 		rq->rt_avg /= 2;
 	}
 }