linux-kernel - Re: RCU related performance regression in 3.3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 27 Apr 2012 14:15:20 +0200 (CEST)
From:	Pascal Chapperon <pascal.chapperon@...adoo.fr>
To:	paulmck@...ux.vnet.ibm.com
Cc:	Josh Boyer <jwboyer@...hat.com>, linux-kernel@...r.kernel.org,
	kernel-team@...oraproject.org
Subject: Re: RCU related performance regression in 3.3

Le 18/04/2012 17:23, Paul E. McKenney a écrit :
> On Wed, Apr 18, 2012 at 05:00:14PM +0200, Pascal Chapperon wrote:
>> Le 18/04/2012 16:01, Paul E. McKenney a écrit :
>>> On Wed, Apr 18, 2012 at 11:37:28AM +0200, Pascal Chapperon wrote:
>>>> Mount and umount operations are not slower with RCU_FAST_NO_HZ during
>>>> runtime; systemctl start and stop operations are also not slower. In
>>>> fact, i couldn't find a single operation slower during runtime with
>>>> RCU_FAST_NO_HZ.
>>>
>>> Your boot-time setup is such that all CPUs are online before the
>>> boot-time mount operations take place, right?
>> Yes :
>> [ 0.242697] Brought up 8 CPUs
>> [ 0.242699] Total of 8 processors activated (35118.33 BogoMIPS).
>>
>>> Struggling to understand
>>> how RCU can tell the difference between post-CPU-bringup boot time
>>> and run time...
>>>
>> systemd is controlling the whole boot process including mount
>> operation (apart root filesystem) and as I can see, uses heavily
>> sockets to do it (not to mention cpu-affinity). It controls also the
>> major part of umount operations. Is it possible that your patch hits
>> a systemd bug ?
> 
> Is it possible that systemd is using network operations that include
> synchronize_rcu()? Then if you did the same operation from the
> command line at runtime, you might not see the slowdown.
> 
> Is it possible for you to convince systemd to collect RCU event tracing
> during the slow operation? RCU event tracing is available under
> /sys/kernel/debug/tracing/rcu.
>
.
I have collected the RCU event tracing during a slow boot with
FAST_NO_HZ (and the same without FAST_NO_HZ, same kernel config).
The full logs and associated "systemd-analyze plot" can be found
(in comment 32) at :

https://bugzilla.redhat.com/show_bug.cgi?id=806548

With FAST_NO_HZ, almost each rcu_prep_idle is followed by ksoftirqd
(75000 ksoftirqd lines with FAST_NO_HZ, 4000 without).

Sorry, the logs are very huge, but I can't figure where are the 
plots of some interest. 

Pascal
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/