linux-kernel - [tip:locking/core] locking/qspinlock: Revert to test-and-set on hypervisors

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <tip-2aa79af64263190eec610422b07f60e99a7d230a@git.kernel.org>
Date:	Fri, 8 May 2015 06:27:21 -0700
From:	"tip-bot for Peter Zijlstra (Intel)" <tipbot@...or.com>
To:	linux-tip-commits@...r.kernel.org
Cc:	tglx@...utronix.de, mingo@...nel.org, oleg@...hat.com,
	hpa@...or.com, raghavendra.kt@...ux.vnet.ibm.com,
	daniel@...ascale.com, akpm@...ux-foundation.org,
	paolo.bonzini@...il.com, david.vrabel@...rix.com,
	linux-kernel@...r.kernel.org, riel@...hat.com, doug.hatch@...com,
	konrad.wilk@...cle.com, scott.norton@...com,
	boris.ostrovsky@...cle.com, bp@...en8.de,
	torvalds@...ux-foundation.org, paulmck@...ux.vnet.ibm.com,
	Waiman.Long@...com, peterz@...radead.org
Subject: [tip:locking/core] locking/qspinlock:
  Revert to test-and-set on hypervisors

Commit-ID:  2aa79af64263190eec610422b07f60e99a7d230a
Gitweb:     http://git.kernel.org/tip/2aa79af64263190eec610422b07f60e99a7d230a
Author:     Peter Zijlstra (Intel) <peterz@...radead.org>
AuthorDate: Fri, 24 Apr 2015 14:56:36 -0400
Committer:  Ingo Molnar <mingo@...nel.org>
CommitDate: Fri, 8 May 2015 12:36:58 +0200

locking/qspinlock: Revert to test-and-set on hypervisors

When we detect a hypervisor (!paravirt, see qspinlock paravirt support
patches), revert to a simple test-and-set lock to avoid the horrors
of queue preemption.

Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
Signed-off-by: Waiman Long <Waiman.Long@...com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
Cc: Andrew Morton <akpm@...ux-foundation.org>
Cc: Boris Ostrovsky <boris.ostrovsky@...cle.com>
Cc: Borislav Petkov <bp@...en8.de>
Cc: Daniel J Blueman <daniel@...ascale.com>
Cc: David Vrabel <david.vrabel@...rix.com>
Cc: Douglas Hatch <doug.hatch@...com>
Cc: H. Peter Anvin <hpa@...or.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Oleg Nesterov <oleg@...hat.com>
Cc: Paolo Bonzini <paolo.bonzini@...il.com>
Cc: Paul E. McKenney <paulmck@...ux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Raghavendra K T <raghavendra.kt@...ux.vnet.ibm.com>
Cc: Rik van Riel <riel@...hat.com>
Cc: Scott J Norton <scott.norton@...com>
Cc: Thomas Gleixner <tglx@...utronix.de>
Cc: virtualization@...ts.linux-foundation.org
Cc: xen-devel@...ts.xenproject.org
Link: http://lkml.kernel.org/r/1429901803-29771-8-git-send-email-Waiman.Long@hp.com
Signed-off-by: Ingo Molnar <mingo@...nel.org>
---
 arch/x86/include/asm/qspinlock.h | 14 ++++++++++++++
 include/asm-generic/qspinlock.h  |  7 +++++++
 kernel/locking/qspinlock.c       |  3 +++
 3 files changed, 24 insertions(+)

diff --git a/arch/x86/include/asm/qspinlock.h b/arch/x86/include/asm/qspinlock.h
index e2aee82..f079b70 100644
--- a/arch/x86/include/asm/qspinlock.h
+++ b/arch/x86/include/asm/qspinlock.h
@@ -1,6 +1,7 @@
 #ifndef _ASM_X86_QSPINLOCK_H
 #define _ASM_X86_QSPINLOCK_H
 
+#include <asm/cpufeature.h>
 #include <asm-generic/qspinlock_types.h>
 
 #define	queued_spin_unlock queued_spin_unlock
@@ -15,6 +16,19 @@ static inline void queued_spin_unlock(struct qspinlock *lock)
 	smp_store_release((u8 *)lock, 0);
 }
 
+#define virt_queued_spin_lock virt_queued_spin_lock
+
+static inline bool virt_queued_spin_lock(struct qspinlock *lock)
+{
+	if (!static_cpu_has(X86_FEATURE_HYPERVISOR))
+		return false;
+
+	while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0)
+		cpu_relax();
+
+	return true;
+}
+
 #include <asm-generic/qspinlock.h>
 
 #endif /* _ASM_X86_QSPINLOCK_H */
diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinlock.h
index 569abcd..83bfb87 100644
--- a/include/asm-generic/qspinlock.h
+++ b/include/asm-generic/qspinlock.h
@@ -111,6 +111,13 @@ static inline void queued_spin_unlock_wait(struct qspinlock *lock)
 		cpu_relax();
 }
 
+#ifndef virt_queued_spin_lock
+static __always_inline bool virt_queued_spin_lock(struct qspinlock *lock)
+{
+	return false;
+}
+#endif
+
 /*
  * Initializier
  */
diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
index 0338721..fd31a47 100644
--- a/kernel/locking/qspinlock.c
+++ b/kernel/locking/qspinlock.c
@@ -249,6 +249,9 @@ void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
 
 	BUILD_BUG_ON(CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS));
 
+	if (virt_queued_spin_lock(lock))
+		return;
+
 	/*
 	 * wait for in-progress pending->locked hand-overs
 	 *
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/