linux-kernel - Re: tree rcu: call_rcu scalability problem?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090903071416.GB979@wotan.suse.de>
Date:	Thu, 3 Sep 2009 09:14:16 +0200
From:	Nick Piggin <npiggin@...e.de>
To:	Peter Zijlstra <peterz@...radead.org>
Cc:	Paul McKenney <paulmck@...ibm.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: tree rcu: call_rcu scalability problem?

On Wed, Sep 02, 2009 at 09:17:44PM +0200, Peter Zijlstra wrote:
> On Wed, 2009-09-02 at 14:27 +0200, Nick Piggin wrote:
> 
> > It seems like nearly 2/3 of the cost is here:
> >         /* Add the callback to our list. */
> >         *rdp->nxttail[RCU_NEXT_TAIL] = head; <<<
> >         rdp->nxttail[RCU_NEXT_TAIL] = &head->next;
> > 
> > In loading the pointer to the next tail pointer. If I'm reading the profile
> > correctly. Can't see why that should be a probem though...
> > 
> > ffffffff8107dee0 <__call_rcu>: /* __call_rcu total: 320971 100.000 */
> >    697  0.2172 :ffffffff8107dee0:       push   %r12
> 
> >    921  0.2869 :ffffffff8107df57:       push   %rdx
> >    151  0.0470 :ffffffff8107df58:       popfq
> > 183507 57.1725 :ffffffff8107df59:       mov    0x50(%rbx),%rax
> >    995  0.3100 :ffffffff8107df5d:       mov    %rdi,(%rax)
> 
> I'd guess at popfq to be the expensive op here.. skid usually causes the
> attribution to be a few ops down the line.

Well it doesn't really explain why it is getting more costly per
unit of work as core count increases. The popfq should run completely
out of L1 cache (or even from store forwarding) and it is expensive,
but those costs should remain the same regardless of how many cores
in the system.

But I am getting some strange looking profiles, so I'm going to get
a bigger system to run it on to see if we can get really pronounced
scalability problem. Maybe even make a microbenchmark to eliminate
other elements of the workload.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/