linux-kernel - [RFC PATCH 3/7] locking/rtqspinlock: Use static RT priority when in interrupt context

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue,  3 Jan 2017 13:00:26 -0500
From:   Waiman Long <longman@...hat.com>
To:     Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        "H. Peter Anvin" <hpa@...or.com>
Cc:     linux-kernel@...r.kernel.org, Waiman Long <longman@...hat.com>
Subject: [RFC PATCH 3/7] locking/rtqspinlock: Use static RT priority when in interrupt context

When in interrupt context, the priority of the interrupted task is
meaningless. So static RT priority is assigned in this case to make
sure that it can get lock ASAP to reduce latency to the interrupted
task.

Signed-off-by: Waiman Long <longman@...hat.com>
---
 kernel/locking/qspinlock_rt.h | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/kernel/locking/qspinlock_rt.h b/kernel/locking/qspinlock_rt.h
index 69513d6..b6289fb 100644
--- a/kernel/locking/qspinlock_rt.h
+++ b/kernel/locking/qspinlock_rt.h
@@ -19,6 +19,13 @@
  *
  * As RT qspinlock needs the whole pending byte, it cannot be used on kernel
  * configured to support 16K or more CPUs (CONFIG_NR_CPUS).
+ *
+ * In interrupt context, the priority of the interrupted task is not
+ * meaningful. So a fixed static RT priority is used and they won't go into
+ * the MCS wait queue.
+ *  1) Soft IRQ = 1
+ *  2) Hard IRQ = MAX_RT_PRIO
+ *  3) NMI	= MAX_RT_PRIO+1
  */
 #include <linux/sched.h>
 
@@ -60,11 +67,15 @@ static inline int rt_pending(int val)
 static bool rt_spin_trylock(struct qspinlock *lock)
 {
 	struct __qspinlock *l = (void *)lock;
-	u8 prio = rt_task_priority(current);
+	struct task_struct *task = in_interrupt() ? NULL : current;
+	u8 prio;
 
 	BUILD_BUG_ON(_Q_PENDING_BITS != 8);
 
-	if (!prio)
+	if (!task)
+		prio = in_nmi() ? MAX_RT_PRIO + 1
+		     : in_irq() ? MAX_RT_PRIO : 1;
+	else if (!(prio = rt_task_priority(task)))
 		return false;
 
 	/*
-- 
1.8.3.1