linux-kernel - [RFC PATCH 5/6] timers: Introduce soft-interruptible timers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue,  1 Aug 2023 15:24:40 +0200
From:   Frederic Weisbecker <frederic@...nel.org>
To:     LKML <linux-kernel@...r.kernel.org>
Cc:     Frederic Weisbecker <frederic@...nel.org>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        "Paul E . McKenney" <paulmck@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Anna-Maria Behnsen <anna-maria@...utronix.de>,
        Eric Dumazet <edumazet@...gle.com>
Subject: [RFC PATCH 5/6] timers: Introduce soft-interruptible timers

Most timers are unrelated to networking or other softirq vectors (RCU,
NET_*, HRTIMER, TASKLET, ...). Yet when a timer batch is executing,
other softirq vectors have to wait for the timers batch completion
even though there is nothing to synchronize against most callbacks.

However there is no automatic way to determine if a timer callback is
safely soft-interruptible by other vectors. So the only long term viable
approach to solve this is to adopt a progressive push down solution
similar to the one used for getting rid of the big kernel lock.

Introduce a new TIMER_SOFTINTERRUPTIBLE flag which tells the timer
subsystem that a callback is safely soft-interruptible by other vectors,
either because it's completely unrelated to them or because it uses the
appropriate local_bh_disable()/spin_lock_bh() on narrowed-down regions.

Once all timers are dealt with after a few years, it will become
possible to run timers out of the softirqs processing.

It's worth noting though that if the softirq infrastructure supports
soft-interruption of a TIMER_SOFTINTERRUPTIBLE timer, it doesn't allow
yet a TIMER_SOFTINTERRUPTIBLE timer to soft-interrupt other vectors,
even though nothing prevents from it to happen from a correctness point
of view, more tweaks are needed to support that.

Signed-off-by: Frederic Weisbecker <frederic@...nel.org>
---
 include/linux/timer.h |  5 +++--
 kernel/time/timer.c   | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/include/linux/timer.h b/include/linux/timer.h
index 9162f275819a..fbe40bacc8c3 100644
--- a/include/linux/timer.h
+++ b/include/linux/timer.h
@@ -61,13 +61,14 @@ struct timer_list {
  * should be placed on a particular CPU, then add_timer_on() has to be
  * used.
  */
-#define TIMER_CPUMASK		0x0003FFFF
+#define TIMER_CPUMASK		0x0001FFFF /* If 1 more bit is needed, flags must be 64 */
+#define TIMER_SOFTINTERRUPTIBLE	0x00020000
 #define TIMER_MIGRATING		0x00040000
 #define TIMER_BASEMASK		(TIMER_CPUMASK | TIMER_MIGRATING)
 #define TIMER_DEFERRABLE	0x00080000
 #define TIMER_PINNED		0x00100000
 #define TIMER_IRQSAFE		0x00200000
-#define TIMER_INIT_FLAGS	(TIMER_DEFERRABLE | TIMER_PINNED | TIMER_IRQSAFE)
+#define TIMER_INIT_FLAGS	(TIMER_SOFTINTERRUPTIBLE | TIMER_DEFERRABLE | TIMER_PINNED | TIMER_IRQSAFE)
 #define TIMER_ARRAYSHIFT	22
 #define TIMER_ARRAYMASK		0xFFC00000

diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index 7cad6fe3c035..1e43f54def0e 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -1676,6 +1676,7 @@ static void call_timer_fn(struct timer_list *timer,
 			  unsigned long baseclk)
 {
 	int count = preempt_count();
+	bool softinterruptible = false;

 #ifdef CONFIG_LOCKDEP
 	/*
@@ -1689,6 +1690,17 @@ static void call_timer_fn(struct timer_list *timer,

 	lockdep_copy_map(&lockdep_map, &timer->lockdep_map);
 #endif
+
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) &&
+	    IS_ENABLED(CONFIG_ARCH_HAS_SOFTIRQ_DISABLED_MASK) &&
+	    timer->flags & TIMER_SOFTINTERRUPTIBLE)
+		softinterruptible = true;
+
+	if (softinterruptible) {
+		local_bh_vec_disable(1 << TIMER_SOFTIRQ);
+		local_bh_exit();
+	}
+
 	/*
 	 * Couple the lock chain with the lock chain at
 	 * timer_delete_sync() by acquiring the lock_map around the fn()
@@ -1702,6 +1714,12 @@ static void call_timer_fn(struct timer_list *timer,

 	lock_map_release(&lockdep_map);

+	if (softinterruptible) {
+		local_bh_enter();
+		local_bh_vec_enable(1 << TIMER_SOFTIRQ);
+	}
+
+
 	if (count != preempt_count()) {
 		WARN_ONCE(1, "timer: %pS preempt leak: %08x -> %08x\n",
 			  fn, count, preempt_count());
-- 
2.34.1