linux-kernel - [tip:timers/urgent] x86/timers/apic: Fix imprecise timer interrupts by eliminating TSC clockevents frequency roundoff error

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <tip-1a9e4c564ab174e53ed86def922804a5ddc63e7d@git.kernel.org>
Date:	Wed, 10 Aug 2016 10:55:23 -0700
From:	tip-bot for Nicolai Stange <tipbot@...or.com>
To:	linux-tip-commits@...r.kernel.org
Cc:	bp@...e.de, len.brown@...el.com, torvalds@...ux-foundation.org,
	hidehiro.kawai.ez@...achi.com, christopher.s.hall@...el.com,
	pbonzini@...hat.com, peterz@...radead.org, hpa@...or.com,
	tglx@...utronix.de, mingo@...nel.org, nicstange@...il.com,
	linux-kernel@...r.kernel.org, viresh.kumar@...aro.org,
	adrian.hunter@...el.com
Subject: [tip:timers/urgent] x86/timers/apic: Fix imprecise timer interrupts
 by eliminating TSC clockevents frequency roundoff error

Commit-ID:  1a9e4c564ab174e53ed86def922804a5ddc63e7d
Gitweb:     http://git.kernel.org/tip/1a9e4c564ab174e53ed86def922804a5ddc63e7d
Author:     Nicolai Stange <nicstange@...il.com>
AuthorDate: Thu, 14 Jul 2016 17:22:54 +0200
Committer:  Ingo Molnar <mingo@...nel.org>
CommitDate: Wed, 10 Aug 2016 12:37:38 +0200

x86/timers/apic: Fix imprecise timer interrupts by eliminating TSC clockevents frequency roundoff error

I noticed the following bug/misbehavior on certain Intel systems: with a
single task running on a NOHZ CPU on an Intel Haswell, I recognized
that I did not only get the one expected local_timer APIC interrupt, but
two per second at minimum. (!)

Further tracing showed that the first one precedes the programmed deadline
by up to ~50us and hence, it did nothing except for reprogramming the TSC
deadline clockevent device to trigger shortly thereafter again.

The reason for this is imprecise calibration, the timeout we program into
the APIC results in 'too short' timer interrupts. The core (hr)timer code
notices this (because it has a precise ktime source and sees the short
interrupt) and fixes it up by programming an additional very short
interrupt period.

This is obviously suboptimal.

The reason for the imprecise calibration is twofold, and this patch
fixes the first reason:

In setup_APIC_timer(), the registered clockevent device's frequency
is calculated by first dividing tsc_khz by TSC_DIVISOR and multiplying
it with 1000 afterwards:

  (tsc_khz / TSC_DIVISOR) * 1000

The multiplication with 1000 is done for converting from kHz to Hz and the
division by TSC_DIVISOR is carried out in order to make sure that the final
result fits into an u32.

However, with the order given in this calculation, the roundoff error
introduced by the division gets magnified by a factor of 1000 by the
following multiplication.

To fix it, reversing the order of the division and the multiplication a la:

  (tsc_khz * 1000) / TSC_DIVISOR

... reduces the roundoff error already.

Furthermore, if TSC_DIVISOR divides 1000, associativity holds:

  (tsc_khz * 1000) / TSC_DIVISOR = tsc_khz * (1000 / TSC_DIVISOR)

and thus, the roundoff error even vanishes and the whole operation can be
carried out within 32 bits.

The powers of two that divide 1000 are 2, 4 and 8. A value of 8 for
TSC_DIVISOR still allows for TSC frequencies up to
2^32 / 10^9ns * 8 = 34.4GHz which is way larger than anything to expect
in the next years.

Thus we also replace the current TSC_DIVISOR value of 32 by 8. Reverse
the order of the divison and the multiplication in the calculation of
the registered clockevent device's frequency.

Signed-off-by: Nicolai Stange <nicstange@...il.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
Reviewed-by: Paolo Bonzini <pbonzini@...hat.com>
Acked-by: Thomas Gleixner <tglx@...utronix.de>
Cc: Adrian Hunter <adrian.hunter@...el.com>
Cc: Borislav Petkov <bp@...e.de>
Cc: Christopher S. Hall <christopher.s.hall@...el.com>
Cc: H. Peter Anvin <hpa@...or.com>
Cc: Hidehiro Kawai <hidehiro.kawai.ez@...achi.com>
Cc: Len Brown <len.brown@...el.com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Viresh Kumar <viresh.kumar@...aro.org>
Link: http://lkml.kernel.org/r/20160714152255.18295-2-nicstange@gmail.com
[ Improved changelog. ]
Signed-off-by: Ingo Molnar <mingo@...nel.org>
---
 arch/x86/kernel/apic/apic.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index ac8d8ad..a315dc4 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -313,7 +313,7 @@ int lapic_get_maxlvt(void)

 /* Clock divisor */
 #define APIC_DIVISOR 16
-#define TSC_DIVISOR  32
+#define TSC_DIVISOR  8

 /*
  * This function sets up the local APIC timer, with a timeout of
@@ -565,7 +565,7 @@ static void setup_APIC_timer(void)
 				    CLOCK_EVT_FEAT_DUMMY);
 		levt->set_next_event = lapic_next_deadline;
 		clockevents_config_and_register(levt,
-						(tsc_khz / TSC_DIVISOR) * 1000,
+						tsc_khz * (1000 / TSC_DIVISOR),
 						0xF, ~0UL);
 	} else
 		clockevents_register_device(levt);