| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CACT4Y+ZtjkwnGwsLGytvN7Zy+O8o_iewUAP5FK5vun34kdDScA@mail.gmail.com>
Date: Thu, 12 May 2022 10:04:24 +0200
From: Dmitry Vyukov <dvyukov@...gle.com>
To: Ian Rogers <irogers@...gle.com>
Cc: peterz@...radead.org, mingo@...hat.com, acme@...nel.org,
mark.rutland@....com, alexander.shishkin@...ux.intel.com,
jolsa@...nel.org, namhyung@...nel.org, elver@...gle.com,
linux-perf-users@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] tools/perf: add breakpoint benchmarks
On Wed, 11 May 2022 at 17:35, Ian Rogers <irogers@...gle.com> wrote:
>
> On Thu, May 5, 2022 at 8:58 AM Dmitry Vyukov <dvyukov@...gle.com> wrote:
> >
> > Add 2 benchmarks:
> > 1. Performance of thread creation/exiting in presence of breakpoints.
> > 2. Performance of breakpoint modification in presence of threads.
> >
> > The benchmarks capture use cases that we are interested in:
> > using inheritable breakpoints in large highly-threaded applications.
> > The benchmarks show significant slowdown imposed by breakpoints
> > (even when they don't fire).
> >
> > Testing on Intel 8173M with 112 HW threads show:
> >
> > perf bench --repeat=56 breakpoint thread --breakpoints=0 --parallelism=56 --threads=20
> > 78.675000 usecs/op
> > perf bench --repeat=56 breakpoint thread --breakpoints=4 --parallelism=56 --threads=20
> > 12967.135714 usecs/op
> > That's 165x slowdown due to presence of the breakpoints.
> >
> > perf bench --repeat=20000 breakpoint enable --passive=0 --active=0
> > 1.433250 usecs/op
> > perf bench --repeat=20000 breakpoint enable --passive=224 --active=0
> > 585.318400 usecs/op
> > perf bench --repeat=20000 breakpoint enable --passive=0 --active=111
> > 635.953000 usecs/op
> > That's 408x and 444x slowdown due to presence of threads.
> >
> > Profiles show some overhead in toggle_bp_slot,
> > but also very high contention:
> >
> > 90.83% breakpoint-thre [kernel.kallsyms] [k] osq_lock
> > 4.69% breakpoint-thre [kernel.kallsyms] [k] mutex_spin_on_owner
> > 2.06% breakpoint-thre [kernel.kallsyms] [k] __reserve_bp_slot
> > 2.04% breakpoint-thre [kernel.kallsyms] [k] toggle_bp_slot
> >
> > 79.01% breakpoint-enab [kernel.kallsyms] [k] smp_call_function_single
> > 9.94% breakpoint-enab [kernel.kallsyms] [k] llist_add_batch
> > 5.70% breakpoint-enab [kernel.kallsyms] [k] _raw_spin_lock_irq
> > 1.84% breakpoint-enab [kernel.kallsyms] [k] event_function_call
> > 1.12% breakpoint-enab [kernel.kallsyms] [k] send_call_function_single_ipi
> > 0.37% breakpoint-enab [kernel.kallsyms] [k] generic_exec_single
> > 0.24% breakpoint-enab [kernel.kallsyms] [k] __perf_event_disable
> > 0.20% breakpoint-enab [kernel.kallsyms] [k] _perf_event_enable
> > 0.18% breakpoint-enab [kernel.kallsyms] [k] toggle_bp_slot
> >
> > Signed-off-by: Dmitry Vyukov <dvyukov@...gle.com>
>
> Acked-by: Ian Rogers <irogers@...gle.com>
>
> > Cc: linux-perf-users@...r.kernel.org
> > Cc: linux-kernel@...r.kernel.org
> > ---
> > tools/perf/bench/Build | 1 +
> > tools/perf/bench/bench.h | 2 +
> > tools/perf/bench/breakpoint.c | 235 ++++++++++++++++++++++++++++++++++
> > tools/perf/builtin-bench.c | 8 ++
> > 4 files changed, 246 insertions(+)
> > create mode 100644 tools/perf/bench/breakpoint.c
> >
> > diff --git a/tools/perf/bench/Build b/tools/perf/bench/Build
> > index 61d45fcb4057c..6b6155a8ad096 100644
> > --- a/tools/perf/bench/Build
> > +++ b/tools/perf/bench/Build
> > @@ -14,6 +14,7 @@ perf-y += kallsyms-parse.o
> > perf-y += find-bit-bench.o
> > perf-y += inject-buildid.o
> > perf-y += evlist-open-close.o
> > +perf-y += breakpoint.o
> >
> > perf-$(CONFIG_X86_64) += mem-memcpy-x86-64-asm.o
> > perf-$(CONFIG_X86_64) += mem-memset-x86-64-asm.o
> > diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h
> > index b3480bc33fe84..6cefb4315d75e 100644
> > --- a/tools/perf/bench/bench.h
> > +++ b/tools/perf/bench/bench.h
> > @@ -49,6 +49,8 @@ int bench_synthesize(int argc, const char **argv);
> > int bench_kallsyms_parse(int argc, const char **argv);
> > int bench_inject_build_id(int argc, const char **argv);
> > int bench_evlist_open_close(int argc, const char **argv);
> > +int bench_breakpoint_thread(int argc, const char **argv);
> > +int bench_breakpoint_enable(int argc, const char **argv);
> >
> > #define BENCH_FORMAT_DEFAULT_STR "default"
> > #define BENCH_FORMAT_DEFAULT 0
> > diff --git a/tools/perf/bench/breakpoint.c b/tools/perf/bench/breakpoint.c
> > new file mode 100644
> > index 0000000000000..56936fea246d7
> > --- /dev/null
> > +++ b/tools/perf/bench/breakpoint.c
> > @@ -0,0 +1,235 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include <subcmd/parse-options.h>
> > +#include <linux/hw_breakpoint.h>
> > +#include <linux/perf_event.h>
> > +#include <linux/time64.h>
> > +#include <sys/syscall.h>
> > +#include <sys/ioctl.h>
> > +#include <sys/time.h>
> > +#include <pthread.h>
> > +#include <stddef.h>
> > +#include <stdlib.h>
> > +#include <unistd.h>
> > +#include <stdio.h>
> > +#include <errno.h>
> > +#include "bench.h"
> > +#include "futex.h"
> > +
> > +struct {
> > + unsigned int nbreakpoints;
> > + unsigned int nparallel;
> > + unsigned int nthreads;
> > +} thread_params = {
> > + .nbreakpoints = 1,
> > + .nparallel = 1,
> > + .nthreads = 1,
> > +};
> > +
> > +static const struct option thread_options[] = {
> > + OPT_UINTEGER('b', "breakpoints", &thread_params.nbreakpoints,
> > + "Specify amount of breakpoints"),
> > + OPT_UINTEGER('p', "parallelism", &thread_params.nparallel, "Specify amount of parallelism"),
> > + OPT_UINTEGER('t', "threads", &thread_params.nthreads, "Specify amount of threads"),
> > + OPT_END()
> > +};
> > +
> > +static const char * const thread_usage[] = {
> > + "perf bench breakpoint thread <options>",
> > + NULL
> > +};
> > +
> > +struct breakpoint {
> > + int fd;
> > + char watched;
> > +};
> > +
> > +static int breakpoint_setup(void *addr)
> > +{
> > + struct perf_event_attr attr = {0};
> > +
> > + attr.type = PERF_TYPE_BREAKPOINT;
> > + attr.size = sizeof(attr);
> > + attr.inherit = 1;
> > + attr.exclude_kernel = 1;
> > + attr.exclude_hv = 1;
> > + attr.bp_addr = (uint64_t)addr;
> > + attr.bp_type = HW_BREAKPOINT_RW;
> > + attr.bp_len = HW_BREAKPOINT_LEN_1;
> > + return syscall(SYS_perf_event_open, &attr, 0, -1, -1, 0);
> > +}
> > +
> > +static void *passive_thread(void *arg)
> > +{
> > + unsigned int *done = (unsigned int *)arg;
> > +
> > + while (!__atomic_load_n(done, __ATOMIC_RELAXED))
>
> Note, this may be the first change with atomics since Linux moved to
> C11 which may make the use of stdatomic.h more idiomatic. Just
> flagging for potential future cleanup.
I am ready to send a follow up with stdatomic.h if that's considered a
better option.
Powered by blists - more mailing lists