linux-kernel - [PATCH 2/6] x86: move decision about clearing slowpath flag into arch_spin

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1430391243-7112-3-git-send-email-jgross@suse.com>
Date:	Thu, 30 Apr 2015 12:53:59 +0200
From:	Juergen Gross <jgross@...e.com>
To:	linux-kernel@...r.kernel.org, x86@...nel.org, hpa@...or.com,
	tglx@...utronix.de, mingo@...hat.com,
	xen-devel@...ts.xensource.com, konrad.wilk@...cle.com,
	david.vrabel@...rix.com, boris.ostrovsky@...cle.com,
	jeremy@...p.org, chrisw@...s-sol.org, akataria@...are.com,
	rusty@...tcorp.com.au, virtualization@...ts.linux-foundation.org,
	gleb@...nel.org, pbonzini@...hat.com, kvm@...r.kernel.org
Cc:	Juergen Gross <jgross@...e.com>
Subject: [PATCH 2/6] x86: move decision about clearing slowpath flag into arch_spin_lock()

The decision whether the slowpath flag is to be cleared for
paravirtualized spinlocks is located in
__ticket_check_and_clear_slowpath() today.

Move that decision into arch_spin_lock() and add an unlikely attribute
to it to avoid calling a function in case the compiler chooses not to
inline __ticket_check_and_clear_slowpath() and the slowpath flag isn't
set.

Signed-off-by: Juergen Gross <jgross@...e.com>
---
 arch/x86/include/asm/spinlock.h | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/arch/x86/include/asm/spinlock.h b/arch/x86/include/asm/spinlock.h
index 8ceec8d..268b9da 100644
--- a/arch/x86/include/asm/spinlock.h
+++ b/arch/x86/include/asm/spinlock.h
@@ -66,20 +66,18 @@ static inline int  __tickets_equal(__ticket_t one, __ticket_t two)
 	return !((one ^ two) & ~TICKET_SLOWPATH_FLAG);
 }
 
-static inline void __ticket_check_and_clear_slowpath(arch_spinlock_t *lock,
-							__ticket_t head)
+static inline void __ticket_clear_slowpath(arch_spinlock_t *lock,
+					   __ticket_t head)
 {
-	if (head & TICKET_SLOWPATH_FLAG) {
-		arch_spinlock_t old, new;
+	arch_spinlock_t old, new;
 
-		old.tickets.head = head;
-		new.tickets.head = head & ~TICKET_SLOWPATH_FLAG;
-		old.tickets.tail = new.tickets.head + TICKET_LOCK_INC;
-		new.tickets.tail = old.tickets.tail;
+	old.tickets.head = head;
+	new.tickets.head = head & ~TICKET_SLOWPATH_FLAG;
+	old.tickets.tail = new.tickets.head + TICKET_LOCK_INC;
+	new.tickets.tail = old.tickets.tail;
 
-		/* try to clear slowpath flag when there are no contenders */
-		cmpxchg(&lock->head_tail, old.head_tail, new.head_tail);
-	}
+	/* try to clear slowpath flag when there are no contenders */
+	cmpxchg(&lock->head_tail, old.head_tail, new.head_tail);
 }
 
 static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock)
@@ -120,7 +118,8 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock)
 		__ticket_lock_spinning(lock, inc.tail);
 	}
 clear_slowpath:
-	__ticket_check_and_clear_slowpath(lock, inc.head);
+	if (unlikely(inc.head & TICKET_SLOWPATH_FLAG))
+		__ticket_clear_slowpath(lock, inc.head);
 out:
 	barrier();	/* make sure nothing creeps before the lock is taken */
 }
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/