linux-kernel - Re: [PATCH-tip v7 08/20] locking/rwsem: Implement lock handoff to prevent lock starvation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190503131054.GE2623@hirez.programming.kicks-ass.net>
Date:   Fri, 3 May 2019 15:10:54 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Waiman Long <longman@...hat.com>
Cc:     Ingo Molnar <mingo@...hat.com>, Will Deacon <will.deacon@....com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Borislav Petkov <bp@...en8.de>,
        "H. Peter Anvin" <hpa@...or.com>, linux-kernel@...r.kernel.org,
        x86@...nel.org, Davidlohr Bueso <dave@...olabs.net>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Tim Chen <tim.c.chen@...ux.intel.com>,
        huang ying <huang.ying.caritas@...il.com>
Subject: Re: [PATCH-tip v7 08/20] locking/rwsem: Implement lock handoff to
 prevent lock starvation

On Sun, Apr 28, 2019 at 05:25:45PM -0400, Waiman Long wrote:
> +static inline bool rwsem_try_write_lock(long count, struct rw_semaphore *sem,
> +					enum writer_wait_state wstate)
>  {
>  	long new;
>  
> +	lockdep_assert_held(&sem->wait_lock);
> +	do {
> +		bool has_handoff = !!(count & RWSEM_FLAG_HANDOFF);
>  
> +		if (has_handoff && wstate == WRITER_NOT_FIRST)
> +			return false;
>  
> +		if (count & RWSEM_LOCK_MASK) {
> +			if (has_handoff || (wstate != WRITER_HANDOFF))
> +				return false;
> +			new = count | RWSEM_FLAG_HANDOFF;
> +		} else {
> +			new = (count | RWSEM_WRITER_LOCKED) &
> +				~RWSEM_FLAG_HANDOFF;
>  
> +			if (list_is_singular(&sem->wait_list))
> +				new &= ~RWSEM_FLAG_WAITERS;
> +		}
> +	} while (!atomic_long_try_cmpxchg_acquire(&sem->count, &count, new));
> +
> +	/*
> +	 * We have either acquired the lock with handoff bit cleared or
> +	 * set the handoff bit.
> +	 */
> +	if (new & RWSEM_FLAG_HANDOFF)
> +		return false;
> +
> +	rwsem_set_owner(sem);
> +	return true;
>  }

I've changed that like so.

--- a/kernel/locking/rwsem.c
+++ b/kernel/locking/rwsem.c
@@ -406,19 +406,23 @@ static inline bool rwsem_try_write_lock(
 	long new;
 
 	lockdep_assert_held(&sem->wait_lock);
+
 	do {
 		bool has_handoff = !!(count & RWSEM_FLAG_HANDOFF);
 
 		if (has_handoff && wstate == WRITER_NOT_FIRST)
 			return false;
 
+		new = count;
+
 		if (count & RWSEM_LOCK_MASK) {
 			if (has_handoff || (wstate != WRITER_HANDOFF))
 				return false;
-			new = count | RWSEM_FLAG_HANDOFF;
+
+			new |= RWSEM_FLAG_HANDOFF;
 		} else {
-			new = (count | RWSEM_WRITER_LOCKED) &
-				~RWSEM_FLAG_HANDOFF;
+			new |= RWSEM_WRITER_LOCKED;
+			new &= ~RWSEM_FLAG_HANDOFF;
 
 			if (list_is_singular(&sem->wait_list))
 				new &= ~RWSEM_FLAG_WAITERS;