linux-kernel - Re: [PATCH RESEND v2 1/1] percpu_rw_semaphore: reimplement to not block the readers unnecessarily

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20121110005516.GM2419@linux.vnet.ibm.com>
Date:	Fri, 9 Nov 2012 16:55:16 -0800
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Oleg Nesterov <oleg@...hat.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Mikulas Patocka <mpatocka@...hat.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...e.hu>,
	Srikar Dronamraju <srikar@...ux.vnet.ibm.com>,
	Ananth N Mavinakayanahalli <ananth@...ibm.com>,
	Anton Arapov <anton@...hat.com>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH RESEND v2 1/1] percpu_rw_semaphore: reimplement to not
 block the readers unnecessarily

On Fri, Nov 09, 2012 at 07:10:48PM +0100, Oleg Nesterov wrote:
> On 11/09, Paul E. McKenney wrote:
> >
> > On Fri, Nov 09, 2012 at 04:46:56PM +0100, Oleg Nesterov wrote:
> > > Contrary, I am going to try to add some complications later, so that
> > > it can have more users. In particular, I think it can replace
> > > get_online_cpus/cpu_hotplug_begin, just we need
> > > percpu_down_write_but_dont_deadlock_with_recursive_readers().
> >
> > I must confess that I am a bit concerned about possible scalability
> > bottlenecks in the current get_online_cpus(), so +1 from me on this one.
> 
> OK, thanks...
> 
> And btw percpu_down_write_but_dont_deadlock_with_recursive_readers() is
> trivial, just it needs down_write(rw_sem) "inside" wait_event(), not
> before. But I'm afraid I will never manage to write the comments ;)
> 
> 	static bool xxx(brw)
> 	{
> 		down_write(&brw->rw_sem);

		down_write_trylock()

As you noted in your later email.  Presumably you return false if
the attempt to acquire it fails.

> 		if (!atomic_read(&brw->slow_read_ctr))
> 			return true;
> 
> 		up_write(&brw->rw_sem);
> 		return false;
> 	}
> 
> 	static void __percpu_down_write(struct percpu_rw_semaphore *brw, bool recursive_readers)
> 	{
> 		mutex_lock(&brw->writer_mutex);
> 
> 		synchronize_sched();
> 
> 		atomic_add(clear_fast_ctr(brw), &brw->slow_read_ctr);
> 
> 		if (recursive_readers)  {
> 			wait_event(brw->write_waitq, xxx(brw));

I see what you mean about acquiring brw->rw_sem inside of wait_event().

Cute trick!

The "recursive_readers" is a global initialization-time thing, right?

> 		} else {
> 			down_write(&brw->rw_sem);
> 
> 			wait_event(brw->write_waitq, !atomic_read(&brw->slow_read_ctr));
> 		}
> 	}

Looks like it should work, and would perform and scale nicely even
if we end up having to greatly increase the number of calls to
get_online_cpus().

> Of course, cpu.c still needs .active_writer to allow get_online_cpus()
> under cpu_hotplug_begin(), but this is simple.

Yep, same check as now.

> But first we should do other changes, I think. IMHO we should not do
> synchronize_sched() under mutex_lock() and this will add (a bit) more
> complications. We will see.

Indeed, that does put considerable delay on the writers.  There is always
synchronize_sched_expedited(), I suppose.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/