lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20170424162442.GI3956@linux.vnet.ibm.com>
Date:   Mon, 24 Apr 2017 09:24:42 -0700
From:   "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:     Mike Galbraith <efault@....de>
Cc:     LKML <linux-kernel@...r.kernel.org>
Subject: Re: TREE_SRCU slows hotplug by factor ~16

On Mon, Apr 24, 2017 at 09:35:03AM +0200, Mike Galbraith wrote:
> On Sun, 2017-04-23 at 23:22 -0700, Paul E. McKenney wrote:
> 
> > Could you please collect an ftrace (or whatever) showing the timestamp
> > sequence of calls to synchronize_srcu(), synchronize_srcu_expedited(),
> > and call_srcu() during the execution of the stress script?  If it is easy
> > to do, also the timestamp sequence of returns from synchronize_srcu()
> > and synchronize_srcu_expedited()?
> 
> The first two minutes below config bits, trace_printk([enter/exit]) in
> each function.  If you (unlikely, but..) want the whole trace, holler.
> 
> # RCU Subsystem
> CONFIG_TREE_RCU=y
> CONFIG_RCU_EXPERT=y
> CONFIG_SRCU=y
> # CONFIG_CLASSIC_SRCU is not set
> CONFIG_TREE_SRCU=y
> # CONFIG_TASKS_RCU is not set
> CONFIG_RCU_STALL_COMMON=y
> CONFIG_RCU_FANOUT=64
> CONFIG_RCU_FANOUT_LEAF=16
> # CONFIG_RCU_FAST_NO_HZ is not set
> # CONFIG_TREE_RCU_TRACE is not set
> CONFIG_RCU_KTHREAD_PRIO=0
> CONFIG_RCU_NOCB_CPU=y
> CONFIG_RCU_NOCB_CPU_NONE=y
> # CONFIG_RCU_NOCB_CPU_ZERO is not set
> # CONFIG_RCU_NOCB_CPU_ALL is not set
> # RCU Debugging
> # CONFIG_PROVE_RCU is not set
> # CONFIG_SPARSE_RCU_POINTER is not set
> # CONFIG_RCU_PERF_TEST is not set
> # CONFIG_RCU_TORTURE_TEST is not set
> CONFIG_RCU_CPU_STALL_TIMEOUT=60
> # CONFIG_RCU_TRACE is not set
> # CONFIG_RCU_EQS_DEBUG is not set

Thank you for the data, extremely helpful!  I now know that I must
immediately fix this the hard way rather than trying several simpler
possibilities that, according to your trace data, would be complete
wastes of time.

My initial calculation was actually optimistic.  There was one
exit-to-enter interval at about 80 milliseconds, but the second
longest is 342 microseconds, and the vast majority are under 200
microseconds.

Back to the drawing board!

This will take some time, but I should have a patch for you in a few days,
hopefully sooner.

							Thanx, Paul

> # tracer: nop
> #
> # entries-in-buffer/entries-written: 229332/229332   #P:8
> #
> #                              _-----=> irqs-off
> #                             / _----=> need-resched
> #                            | / _---=> hardirq/softirq
> #                            || / _--=> preempt-depth
> #                            ||| /     delay
> #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
> #              | |       |   ||||       |         |
>  stress-cpu-hotp-2443  [004] ....   144.123608: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.123610: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.123612: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.139528: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.139646: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.139646: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.139648: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.155813: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.155981: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.155982: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.155983: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.172078: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.172234: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.172235: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.172237: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.188319: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.188465: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.188466: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.188468: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.202528: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.202650: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.202651: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.202652: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.219081: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.219238: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.219238: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.219240: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.235319: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.235567: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.235568: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.235571: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.251548: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.251706: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.251706: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.251708: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.267865: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.268132: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.268133: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.268136: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.284078: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.284257: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.284258: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.284260: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.300314: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.300462: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.300463: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.300464: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.314533: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.314657: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.314658: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.314659: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.331056: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.331227: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.331227: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.331229: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.347334: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.347517: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.347518: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.347519: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.363567: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.363731: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.363732: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.363733: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.379804: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.379987: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.379988: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.379989: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.396077: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.396259: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.396260: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.396261: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.412385: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.412653: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.412654: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.412657: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.426647: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.426888: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.426889: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.426891: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.443134: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.443325: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.443325: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.443327: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.459327: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.459521: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.459522: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.459524: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.475553: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.475672: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.475673: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.475674: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.491798: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.491919: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.491920: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.491921: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.508035: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.508160: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.508161: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.508162: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.524301: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.524419: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.524419: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.524421: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.538545: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.538665: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.538666: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.538667: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.555051: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.555174: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.555174: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.555175: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.571290: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.571412: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.571412: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.571413: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.587554: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.587672: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.587673: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.587674: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.603843: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.604008: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.604008: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.604010: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.620087: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.620317: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.620318: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.620321: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.636342: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.636520: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.636521: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.636523: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.650579: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.650737: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.650738: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.650739: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.667084: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.667247: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.667247: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.667249: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.683340: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.683513: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.683514: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.683516: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.699556: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.699675: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.699675: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.699676: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.715841: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.716013: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.716014: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.716015: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.732096: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.732269: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.732270: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.732271: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.748393: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.748641: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.748642: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.748645: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.762639: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.762854: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.762855: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.762857: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.779100: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.779278: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.779279: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.779280: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.795348: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.795525: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.795526: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.795527: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.811583: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.811749: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.811750: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.811751: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.827831: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.827997: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.827998: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.827999: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.844104: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.844281: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.844281: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.844283: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.860339: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.860484: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.860485: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.860487: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.874584: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.874737: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.874737: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.874739: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.891117: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   144.891459: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.891460: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   144.891463: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.907376: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   144.907628: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.907629: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   144.907631: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.923590: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   144.923756: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.923756: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   144.923758: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.939837: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   144.940007: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.940008: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   144.940009: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.956110: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   144.956285: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.956286: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   144.956287: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.972344: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   144.972492: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.972493: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   144.972495: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.986566: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   144.986689: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.986689: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   144.986690: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.003114: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.003292: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.003293: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.003294: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.019350: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.019528: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.019529: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.019531: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.035659: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.035939: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.035940: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.035943: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.051891: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.052106: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.052107: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.052109: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.068125: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.068279: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.068279: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.068280: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.084332: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.084472: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.084473: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.084474: call_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   145.098605: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [002] ....   145.098743: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   145.098744: call_srcu: enter
>  stress-cpu-hotp-2443  [002] ....   145.098745: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.114590: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.114728: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.114728: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.114729: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.130591: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.130728: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.130728: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.130730: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.147857: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.228078: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.228079: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.228080: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.244025: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.244106: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.244106: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.244107: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.260286: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.260361: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.260361: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.260362: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.274522: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.274594: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.274594: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.274595: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.291289: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.291358: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.291358: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.291359: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.307529: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.307599: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.307599: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.307600: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.323901: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.324063: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.324063: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.324065: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.340122: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.340288: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.340289: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.340291: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.356378: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.356519: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.356520: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.356521: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.370650: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.370797: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.370798: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.370799: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.387385: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.387547: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.387548: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.387549: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.403636: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.403799: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.403800: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.403802: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.419885: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.420055: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.420056: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.420057: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.436154: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.436372: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.436372: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.436374: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.452413: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.452550: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.452551: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.452552: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.466606: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.466721: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.466721: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.466722: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.483395: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.483555: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.483555: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.483557: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.499628: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.499761: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] .N..   145.499774: call_srcu: enter
>  stress-cpu-hotp-2443  [005] .N..   145.499775: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.515879: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.516009: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.516009: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.516011: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.532164: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.532488: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.532489: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.532492: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.548447: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.548659: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.548660: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.548663: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.562706: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.562897: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.562898: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.562900: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.579403: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.579562: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.579563: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.579564: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.595636: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.595766: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.595766: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.595768: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.611902: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.612059: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.612060: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.612061: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.628136: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.628266: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.628267: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.628268: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.644392: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.644518: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.644519: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.644521: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.658609: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.658698: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.658699: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.658700: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.675410: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.675570: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.675571: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.675572: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.691706: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.691950: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.691951: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.691953: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.707926: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.708096: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.708097: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.708099: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.724154: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.724333: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.724333: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.724335: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.740412: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.740559: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.740559: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.740561: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.754685: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.754835: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.754836: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.754838: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.771415: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.771591: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.771592: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.771594: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.787635: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.787747: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.787748: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.787749: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.803943: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.804168: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.804170: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.804172: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.820168: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.820320: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.820321: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.820322: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.836412: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.836543: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.836544: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.836545: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.850638: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.850757: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.850757: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.850758: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.867433: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.867597: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.867598: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.867599: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.883680: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.883843: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.883843: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.883845: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.899908: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.900073: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.900073: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.900075: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.916213: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   145.916405: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.916406: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   145.916408: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.932453: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   145.932653: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.932653: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   145.932656: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.946674: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   145.946818: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.946819: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   145.946820: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.963402: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   145.963539: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.963540: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   145.963542: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.979671: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   145.979790: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.979790: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   145.979792: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.995927: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   145.996046: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.996046: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   145.996048: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   146.012167: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   146.012282: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   146.012283: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   146.012284: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   146.028405: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   146.028522: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   146.028522: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   146.028523: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   146.042653: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   146.042774: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   146.042774: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   146.042775: call_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   146.059407: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [004] ....   146.059521: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   146.059522: call_srcu: enter
>  stress-cpu-hotp-2443  [004] ....   146.059523: call_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   146.075658: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [005] ....   146.075772: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   146.075773: call_srcu: enter
>  stress-cpu-hotp-2443  [005] ....   146.075774: call_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   146.091906: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [006] ....   146.092021: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   146.092022: call_srcu: enter
>  stress-cpu-hotp-2443  [006] ....   146.092023: call_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   146.108187: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [007] ....   146.108336: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   146.108337: call_srcu: enter
>  stress-cpu-hotp-2443  [007] ....   146.108338: call_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   146.124439: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [000] ....   146.124572: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   146.124572: call_srcu: enter
>  stress-cpu-hotp-2443  [000] ....   146.124574: call_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   146.138656: synchronize_srcu: exit
>  stress-cpu-hotp-2443  [003] ....   146.138773: synchronize_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   146.138773: call_srcu: enter
>  stress-cpu-hotp-2443  [003] ....   146.138774: call_srcu: exit
> 
> 

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ