linux-kernel - Re: [PATCH 3/6] locking/rwsem: Rework writer wakeup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Y/yGZgz1cJ1+pTt5@hirez.programming.kicks-ass.net>
Date:   Mon, 27 Feb 2023 11:31:02 +0100
From:   Peter Zijlstra <peterz@...radead.org>
To:     Waiman Long <longman@...hat.com>
Cc:     mingo@...hat.com, will@...nel.org, linux-kernel@...r.kernel.org,
        boqun.feng@...il.com
Subject: Re: [PATCH 3/6] locking/rwsem: Rework writer wakeup

On Sun, Feb 26, 2023 at 07:22:47PM -0500, Waiman Long wrote:

> @@ -1151,55 +1154,39 @@ rwsem_down_write_slowpath(struct rw_semaphore *sem,
> int state)
>                 }
>         } else {
>                 atomic_long_or(RWSEM_FLAG_WAITERS, &sem->count);
> +               if (rwsem_try_write_lock(sem, &waiter))
> +                       waiter.task = NULL;
>         }
> +       raw_spin_unlock_irq(&sem->wait_lock);
> 
>         /* wait until we successfully acquire the lock */
> -       set_current_state(state);
>         trace_contention_begin(sem, LCB_F_WRITE);
> 
>         for (;;) {
> -               if (rwsem_try_write_lock(sem, &waiter)) {
> -                       /* rwsem_try_write_lock() implies ACQUIRE on success
> */
> +               set_current_state(state);
> +               if (!smp_load_acquire(&waiter.task)) {
> +                       /* Matches rwsem_waiter_wake()'s
> smp_store_release(). */
>                         break;
>                 }
> -
> 
> The additional rwsem_try_write_lock() call seems to address the missed
> wakeup problem AFAICT.

Indeed, prior to this I could readily reproduce the lockup.

So when thinking about missing wakeups I noticed this race on WAITERS.
If we queue but the unlock does not yet observe WAITERS the unlock does
not go into the slow path and wakeup gets lost.

Reader side fixes this with rwsem_cond_wake_waiter(), but I could not
convince myself that is correct for writer side -- perhaps it is, will
need to think more on that.

> I do have some concern that early lock transfer to a lock owner that has not
> been woken up yet may suppress writer lock stealing from optimistic spinning
> causing some performance regression in some cases. Let's see if the test
> robot report anything.

Ah yes, I suppose that is indeed a possibility. Given this is all under
wait_lock and the spinner is not, I was hoping it would still have
sufficient time to win. But yes, robots will tell us.