linux-kernel - RE: [PATCH V3 0/6] export perf overheads information (kernel)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <37D7C6CF3E00A74B8858931C1DB2F077536804E9@SHSMSX103.ccr.corp.intel.com>
Date:   Fri, 16 Dec 2016 15:08:33 +0000
From:   "Liang, Kan" <kan.liang@...el.com>
To:     "peterz@...radead.org" <peterz@...radead.org>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "acme@...nel.org" <acme@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
CC:     "alexander.shishkin@...ux.intel.com" 
        <alexander.shishkin@...ux.intel.com>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "namhyung@...nel.org" <namhyung@...nel.org>,
        "jolsa@...nel.org" <jolsa@...nel.org>,
        "Hunter, Adrian" <adrian.hunter@...el.com>,
        "wangnan0@...wei.com" <wangnan0@...wei.com>,
        "mark.rutland@....com" <mark.rutland@....com>,
        "andi@...stfloor.org" <andi@...stfloor.org>
Subject: RE: [PATCH V3 0/6] export perf overheads information (kernel)


Ping.
Any comments for the series?

Thanks,
Kan

> Subject: [PATCH V3 0/6] export perf overheads information (kernel)
> 
> From: Kan Liang <kan.liang@...el.com>
> 
> This series only include the kernel related patches.
> 
> Profiling brings additional overhead. High overhead may impacts the
> behavior of the profiling object, impacts the accuracy of the profiling result,
> and even hang the system.
> Currently, perf has dynamic interrupt throttle mechanism to lower the
> sample rate and overhead. But it has limitations.
>  - The mechanism only focus in the sampling overhead. However, there
>    are other parts which also bring big overhead. E.g, multiplexing.
>  - The hint from the mechanism doesn't work on fixed period.
>  - The system changes which caused by the mechanism are not recorded
>    in the perf.data. Users have no idea about the overhead and its
>    impact.
> Actually, any passive ways like dynamic interrupt throttle mechanism are
> only palliative. The best way is to export overhead information, provide
> more hints, and help the users design more proper perf command.
> 
> For kernel, three parts can bring obvious overhead.
>   - sample overhead: For x86, it's the time cost in perf NMI handler.
>   - multiplexing overhead: The time cost spends on rotate context.
>   - side-band events overhead: The time cost spends on iterating all
>     events that need to receive side-band events.
> The time cost of those parts are stored in pmu's per-cpu cpuctx.
> The tool can call PERF_EVENT_IOC_STAT when it's 'done'. Then the kernel
> generates the overhead record PERF_RECORD_OVERHEAD.
> 
> User can use the overhead information to refine their perf command and
> get accurate profiling result. For example, if there is high overhead warning,
> user may reduce the number of events/increase the period/decrease the
> frequency.
> Developer can also use the overhead information to find bugs.
> 
> Changes since V2:
>  - Separate kernel patches from the previous version
>  - Add PERF_EVENT_IOC_STAT to control overhead statistics
>  - Collect per pmu overhead information
>  - Store the overhead information in pmu's cpuctx
>  - Add CPU information in overhead record
> 
> Changes since V1:
>  - fix u32 holes and remove duplicate CPU information
>  - configurable overhead logging
>  - Introduce the concept of PMU specific overhead and common core
>    overhead. Rename NMI overhead to PMU sample overhead
>  - Add log_overhead in perf_event_context to indicate the logging of
>    overhead
>  - Refine the output of overhead information
>  - Use perf CPU time to replace perf write data overhead
>  - Refine the formula of overhead evaluation
>  - Refine perf script
> 
> Kan Liang (6):
>   perf/core: Introduce PERF_RECORD_OVERHEAD
>   perf/core: Add PERF_EVENT_IOC_STAT to control overhead statistics
>   perf/x86: implement overhead stat for x86 pmu
>   perf/core: calculate multiplexing overhead
>   perf/core: calculate side-band events overhead
>   perf/x86: calculate sampling overhead
> 
>  arch/x86/events/core.c          | 45 +++++++++++++++++++++++-
>  include/linux/perf_event.h      | 12 +++++++
>  include/uapi/linux/perf_event.h | 44 ++++++++++++++++++++++-
>  kernel/events/core.c            | 77
> ++++++++++++++++++++++++++++++++++++++++-
>  4 files changed, 175 insertions(+), 3 deletions(-)
> 
> --
> 2.4.3