linux-kernel - [tip:locking/core] locking/qspinlock: Handle

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <tip-d682b596d99345ef0000e7017db714ba7f29e017@git.kernel.org>
Date:   Mon, 4 Feb 2019 00:58:14 -0800
From:   tip-bot for Waiman Long <tipbot@...or.com>
To:     linux-tip-commits@...r.kernel.org
Cc:     mingo@...nel.org, will.deacon@....com,
        torvalds@...ux-foundation.org, akpm@...ux-foundation.org,
        longman@...hat.com, bp@...en8.de, peterz@...radead.org,
        hpa@...or.com, srinivas.eeda@...cle.com, tglx@...utronix.de,
        linux-kernel@...r.kernel.org, paulmck@...ux.vnet.ibm.com,
        james.morse@....com, zhenzhong.duan@...cle.com
Subject: [tip:locking/core] locking/qspinlock: Handle > 4 slowpath nesting
 levels

Commit-ID:  d682b596d99345ef0000e7017db714ba7f29e017
Gitweb:     https://git.kernel.org/tip/d682b596d99345ef0000e7017db714ba7f29e017
Author:     Waiman Long <longman@...hat.com>
AuthorDate: Tue, 29 Jan 2019 22:53:45 +0100
Committer:  Ingo Molnar <mingo@...nel.org>
CommitDate: Mon, 4 Feb 2019 09:03:29 +0100

locking/qspinlock: Handle > 4 slowpath nesting levels

Four queue nodes per CPU are allocated to enable up to 4 nesting levels
using the per-CPU nodes. Nested NMIs are possible in some architectures.
Still it is very unlikely that we will ever hit more than 4 nested
levels with contention in the slowpath.

When that rare condition happens, however, it is likely that the system
will hang or crash shortly after that. It is not good and we need to
handle this exception case.

This is done by spinning directly on the lock using repeated trylock.
This alternative code path should only be used when there is nested
NMIs. Assuming that the locks used by those NMI handlers will not be
heavily contended, a simple TAS locking should work out.

Suggested-by: Peter Zijlstra <peterz@...radead.org>
Signed-off-by: Waiman Long <longman@...hat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
Acked-by: Will Deacon <will.deacon@....com>
Cc: Andrew Morton <akpm@...ux-foundation.org>
Cc: Borislav Petkov <bp@...en8.de>
Cc: H. Peter Anvin <hpa@...or.com>
Cc: James Morse <james.morse@....com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Paul E. McKenney <paulmck@...ux.vnet.ibm.com>
Cc: SRINIVAS <srinivas.eeda@...cle.com>
Cc: Thomas Gleixner <tglx@...utronix.de>
Cc: Zhenzhong Duan <zhenzhong.duan@...cle.com>
Link: https://lkml.kernel.org/r/1548798828-16156-2-git-send-email-longman@redhat.com
Signed-off-by: Ingo Molnar <mingo@...nel.org>
---
 kernel/locking/qspinlock.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
index 8a8c3c208c5e..0875053c4050 100644
--- a/kernel/locking/qspinlock.c
+++ b/kernel/locking/qspinlock.c
@@ -412,6 +412,21 @@ pv_queue:
 	idx = node->count++;
 	tail = encode_tail(smp_processor_id(), idx);
 
+	/*
+	 * 4 nodes are allocated based on the assumption that there will
+	 * not be nested NMIs taking spinlocks. That may not be true in
+	 * some architectures even though the chance of needing more than
+	 * 4 nodes will still be extremely unlikely. When that happens,
+	 * we fall back to spinning on the lock directly without using
+	 * any MCS node. This is not the most elegant solution, but is
+	 * simple enough.
+	 */
+	if (unlikely(idx >= MAX_NODES)) {
+		while (!queued_spin_trylock(lock))
+			cpu_relax();
+		goto release;
+	}
+
 	node = grab_mcs_node(node, idx);
 
 	/*