linux-kernel - [tip:locking/urgent] locking/spinlocks/mcs: Micro-optimize osq

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <tip-33ecd2083a9560fbc1ef1b1279ef3ecb4c012a4f@git.kernel.org>
Date:	Wed, 16 Jul 2014 12:21:15 -0700
From:	tip-bot for Jason Low <tipbot@...or.com>
To:	linux-tip-commits@...r.kernel.org
Cc:	mingo@...nel.org, konrad.wilk@...cle.com,
	torvalds@...ux-foundation.org, peterz@...radead.org,
	jason.low2@...com, riel@...hat.com, rostedt@...dmis.org,
	akpm@...ux-foundation.org, tglx@...utronix.de, david@...morbit.com,
	davidlohr@...com, scott.norton@...com,
	linux-kernel@...r.kernel.org, hpa@...or.com, waiman.long@...com,
	tim.c.chen@...ux.intel.com, paulmck@...ux.vnet.ibm.com,
	aswin@...com
Subject: [tip:locking/urgent] locking/spinlocks/mcs:
  Micro-optimize osq_unlock()

Commit-ID:  33ecd2083a9560fbc1ef1b1279ef3ecb4c012a4f
Gitweb:     http://git.kernel.org/tip/33ecd2083a9560fbc1ef1b1279ef3ecb4c012a4f
Author:     Jason Low <jason.low2@...com>
AuthorDate: Mon, 14 Jul 2014 10:27:51 -0700
Committer:  Ingo Molnar <mingo@...nel.org>
CommitDate: Wed, 16 Jul 2014 13:28:06 +0200

locking/spinlocks/mcs: Micro-optimize osq_unlock()

In the unlock function of the cancellable MCS spinlock, the first
thing we do is to retrive the current CPU's osq node. However, due to
the changes made in the previous patch, in the common case where the
lock is not contended, we wouldn't need to access the current CPU's
osq node anymore.

This patch optimizes this by only retriving this CPU's osq node
after we attempt the initial cmpxchg to unlock the osq and found
that its contended.

Signed-off-by: Jason Low <jason.low2@...com>
Signed-off-by: Peter Zijlstra <peterz@...radead.org>
Cc: Scott Norton <scott.norton@...com>
Cc: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
Cc: Dave Chinner <david@...morbit.com>
Cc: Waiman Long <waiman.long@...com>
Cc: Davidlohr Bueso <davidlohr@...com>
Cc: Rik van Riel <riel@...hat.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>
Cc: "H. Peter Anvin" <hpa@...or.com>
Cc: Steven Rostedt <rostedt@...dmis.org>
Cc: Tim Chen <tim.c.chen@...ux.intel.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>
Cc: Aswin Chandramouleeswaran <aswin@...com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Link: http://lkml.kernel.org/r/1405358872-3732-5-git-send-email-jason.low2@hp.com
Signed-off-by: Ingo Molnar <mingo@...nel.org>
---
 kernel/locking/mcs_spinlock.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/locking/mcs_spinlock.c b/kernel/locking/mcs_spinlock.c
index 32fc16c..be9ee15 100644
--- a/kernel/locking/mcs_spinlock.c
+++ b/kernel/locking/mcs_spinlock.c
@@ -182,8 +182,7 @@ unqueue:
 
 void osq_unlock(struct optimistic_spin_queue *lock)
 {
-	struct optimistic_spin_node *node = this_cpu_ptr(&osq_node);
-	struct optimistic_spin_node *next;
+	struct optimistic_spin_node *node, *next;
 	int curr = encode_cpu(smp_processor_id());
 
 	/*
@@ -195,6 +194,7 @@ void osq_unlock(struct optimistic_spin_queue *lock)
 	/*
 	 * Second most likely case.
 	 */
+	node = this_cpu_ptr(&osq_node);
 	next = xchg(&node->next, NULL);
 	if (next) {
 		ACCESS_ONCE(next->locked) = 1;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/