linux-kernel - Re: [PATCH] tools/perf: add breakpoint benchmarks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Wed, 11 May 2022 15:16:58 -0300
From:   Arnaldo Carvalho de Melo <acme@...nel.org>
To:     Ian Rogers <irogers@...gle.com>
Cc:     Dmitry Vyukov <dvyukov@...gle.com>, peterz@...radead.org,
        mingo@...hat.com, mark.rutland@....com,
        alexander.shishkin@...ux.intel.com, jolsa@...nel.org,
        namhyung@...nel.org, elver@...gle.com,
        linux-perf-users@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] tools/perf: add breakpoint benchmarks

Em Wed, May 11, 2022 at 03:08:02PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, May 11, 2022 at 08:34:58AM -0700, Ian Rogers escreveu:
> > On Thu, May 5, 2022 at 8:58 AM Dmitry Vyukov <dvyukov@...gle.com> wrote:
> > >
> > > Add 2 benchmarks:
> > > 1. Performance of thread creation/exiting in presence of breakpoints.
> > > 2. Performance of breakpoint modification in presence of threads.
> > >
> > > The benchmarks capture use cases that we are interested in:
> > > using inheritable breakpoints in large highly-threaded applications.
> > > The benchmarks show significant slowdown imposed by breakpoints
> > > (even when they don't fire).
> > >
> > > Testing on Intel 8173M with 112 HW threads show:
> > >
> > > perf bench --repeat=56 breakpoint thread --breakpoints=0 --parallelism=56 --threads=20
> > >       78.675000 usecs/op
> > > perf bench --repeat=56 breakpoint thread --breakpoints=4 --parallelism=56 --threads=20
> > >    12967.135714 usecs/op
> > > That's 165x slowdown due to presence of the breakpoints.
> > >
> > > perf bench --repeat=20000 breakpoint enable --passive=0 --active=0
> > >        1.433250 usecs/op
> > > perf bench --repeat=20000 breakpoint enable --passive=224 --active=0
> > >      585.318400 usecs/op
> > > perf bench --repeat=20000 breakpoint enable --passive=0 --active=111
> > >      635.953000 usecs/op
> > > That's 408x and 444x slowdown due to presence of threads.
> > >
> > > Profiles show some overhead in toggle_bp_slot,
> > > but also very high contention:
> > >
> > >     90.83%  breakpoint-thre  [kernel.kallsyms]  [k] osq_lock
> > >      4.69%  breakpoint-thre  [kernel.kallsyms]  [k] mutex_spin_on_owner
> > >      2.06%  breakpoint-thre  [kernel.kallsyms]  [k] __reserve_bp_slot
> > >      2.04%  breakpoint-thre  [kernel.kallsyms]  [k] toggle_bp_slot
> > >
> > >     79.01%  breakpoint-enab  [kernel.kallsyms]  [k] smp_call_function_single
> > >      9.94%  breakpoint-enab  [kernel.kallsyms]  [k] llist_add_batch
> > >      5.70%  breakpoint-enab  [kernel.kallsyms]  [k] _raw_spin_lock_irq
> > >      1.84%  breakpoint-enab  [kernel.kallsyms]  [k] event_function_call
> > >      1.12%  breakpoint-enab  [kernel.kallsyms]  [k] send_call_function_single_ipi
> > >      0.37%  breakpoint-enab  [kernel.kallsyms]  [k] generic_exec_single
> > >      0.24%  breakpoint-enab  [kernel.kallsyms]  [k] __perf_event_disable
> > >      0.20%  breakpoint-enab  [kernel.kallsyms]  [k] _perf_event_enable
> > >      0.18%  breakpoint-enab  [kernel.kallsyms]  [k] toggle_bp_slot
> > >
> > > Signed-off-by: Dmitry Vyukov <dvyukov@...gle.com>
> > 
> > Acked-by: Ian Rogers <irogers@...gle.com>
> 
> Thanks, applied.

But I'll add some error checks, etc, running as !root, in a toolbox in
Fedora Silverblue produces:

⬢[acme@...lbox perf]$ perf bench breakpoint all
# Running breakpoint/thread benchmark...
# Created/joined 10 threads with 1 breakpoints and 1 parallelism
     Total time: 0.000 [sec]

      54.600000 usecs/op
      54.600000 usecs/op/cpu

# Running breakpoint/enable benchmark...
# Enabled/disabled breakpoint 10 time with 0 passive and 0 active threads
     Total time: 0.000 [sec]

       1.100000 usecs/op

⬢[acme@...lbox perf]$

⬢[acme@...lbox perf]$ perf bench --repeat=20000 breakpoint enable --passive=224 --active=0
# Running 'breakpoint/enable' benchmark:
# Enabled/disabled breakpoint 20000 time with 224 passive and 0 active threads
     Total time: 8.933 [sec]

     446.674950 usecs/op
⬢[acme@...lbox perf]$

$ grep -m1 "model name" /proc/cpuinfo
model name	: AMD Ryzen 9 5950X 16-Core Processor
 
Diff:

diff --git a/tools/perf/bench/breakpoint.c b/tools/perf/bench/breakpoint.c
index 56936fea246d73c2..d2c074bba06a3d1f 100644
--- a/tools/perf/bench/breakpoint.c
+++ b/tools/perf/bench/breakpoint.c
@@ -83,6 +83,9 @@ static void *breakpoint_thread(void *arg)
 	pthread_t *threads;
 
 	threads = calloc(thread_params.nthreads, sizeof(threads[0]));
+	if (!threads)
+		exit((perror("calloc"), EXIT_FAILURE));
+
 	while (__atomic_fetch_sub(repeat, 1, __ATOMIC_RELAXED) > 0) {
 		done = 0;
 		for (i = 0; i < thread_params.nthreads; i++) {
@@ -114,6 +117,9 @@ int bench_breakpoint_thread(int argc, const char **argv)
 	}
 	breakpoints = calloc(thread_params.nbreakpoints, sizeof(breakpoints[0]));
 	parallel = calloc(thread_params.nparallel, sizeof(parallel[0]));
+	if (!breakpoints || !parallel)
+		exit((perror("calloc"), EXIT_FAILURE));
+
 	for (i = 0; i < thread_params.nbreakpoints; i++) {
 		breakpoints[i].fd = breakpoint_setup(&breakpoints[i].watched);
 		if (breakpoints[i].fd == -1)
@@ -194,6 +200,9 @@ int bench_breakpoint_enable(int argc, const char **argv)
 		exit((perror("perf_event_open"), EXIT_FAILURE));
 	nthreads = enable_params.npassive + enable_params.nactive;
 	threads = calloc(nthreads, sizeof(threads[0]));
+	if (!threads)
+		exit((perror("calloc"), EXIT_FAILURE));
+
 	for (i = 0; i < nthreads; i++) {
 		if (pthread_create(&threads[i], NULL,
 			i < enable_params.npassive ? passive_thread : active_thread, &done))