linux-kernel - Re: RCU related performance regression in 3.3

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120428034257.GA2495@linux.vnet.ibm.com>
Date:	Fri, 27 Apr 2012 20:42:58 -0700
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	Pascal Chapperon <pascal.chapperon@...adoo.fr>
Cc:	Josh Boyer <jwboyer@...hat.com>, linux-kernel@...r.kernel.org,
	kernel-team@...oraproject.org
Subject: Re: RCU related performance regression in 3.3

On Fri, Apr 27, 2012 at 02:15:20PM +0200, Pascal Chapperon wrote:
> Le 18/04/2012 17:23, Paul E. McKenney a écrit :
> > On Wed, Apr 18, 2012 at 05:00:14PM +0200, Pascal Chapperon wrote:
> >> Le 18/04/2012 16:01, Paul E. McKenney a écrit :
> >>> On Wed, Apr 18, 2012 at 11:37:28AM +0200, Pascal Chapperon wrote:
> >>>> Mount and umount operations are not slower with RCU_FAST_NO_HZ during
> >>>> runtime; systemctl start and stop operations are also not slower. In
> >>>> fact, i couldn't find a single operation slower during runtime with
> >>>> RCU_FAST_NO_HZ.
> >>>
> >>> Your boot-time setup is such that all CPUs are online before the
> >>> boot-time mount operations take place, right?
> >> Yes :
> >> [ 0.242697] Brought up 8 CPUs
> >> [ 0.242699] Total of 8 processors activated (35118.33 BogoMIPS).
> >>
> >>> Struggling to understand
> >>> how RCU can tell the difference between post-CPU-bringup boot time
> >>> and run time...
> >>>
> >> systemd is controlling the whole boot process including mount
> >> operation (apart root filesystem) and as I can see, uses heavily
> >> sockets to do it (not to mention cpu-affinity). It controls also the
> >> major part of umount operations. Is it possible that your patch hits
> >> a systemd bug ?
> > 
> > Is it possible that systemd is using network operations that include
> > synchronize_rcu()? Then if you did the same operation from the
> > command line at runtime, you might not see the slowdown.
> > 
> > Is it possible for you to convince systemd to collect RCU event tracing
> > during the slow operation? RCU event tracing is available under
> > /sys/kernel/debug/tracing/rcu.
> >
> .
> I have collected the RCU event tracing during a slow boot with
> FAST_NO_HZ (and the same without FAST_NO_HZ, same kernel config).
> The full logs and associated "systemd-analyze plot" can be found
> (in comment 32) at :
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=806548
> 
> With FAST_NO_HZ, almost each rcu_prep_idle is followed by ksoftirqd
> (75000 ksoftirqd lines with FAST_NO_HZ, 4000 without).
> 
> Sorry, the logs are very huge, but I can't figure where are the 
> plots of some interest. 

Thank you for collecting them!  I clearly will need to do some scripting.  ;-)

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/