linux-kernel - Re: [RFC PATCH] perf script python: Allow reporting [un]throttle

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 31 Aug 2021 14:47:47 -0700
From:   Stephen Brennan <stephen.s.brennan@...cle.com>
To:     Jiri Olsa <jolsa@...hat.com>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Namhyung Kim <namhyung@...nel.org>,
        linux-perf-users@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH] perf script python: Allow reporting [un]throttle

Hi Jiri,

On 8/31/21 11:46 AM, Jiri Olsa wrote:
> On Mon, Aug 16, 2021 at 05:21:33PM -0700, Stephen Brennan wrote:
>> perf_events may sometimes throttle an event due to creating too many
>> samples during a given timer tick. As of now, the perf tool will not
>> report on throttling, which means this is a silent error. Implement a
>> callback for the throttle and unthrottle events within the Python
>> scripting engine, which can allow scripts to detect and report when
>> events may have been lost due to throttling.
>>
>> A python script could simply define throttle() and unthrottle()
>> functions to begin receiving them, e.g.:
>>
>> ```
>> from __future__ import print_function
>>
>> def process_event(param_dict):
>>      print("event cpu={} time={}".format(
>>          param_dict["sample"]["cpu"], param_dict["sample"]["time"]))
>>
>> def throttle(*args):
>>      print("throttle(time={}, cpu={}, pid={}, tid={})".format(*args))
>>
>> def unthrottle(*args):
>>      print("unthrottle(time={}, cpu={}, pid={}, tid={})".format(*args))
>> ```
> 
> throttle event has also 'id' and 'stream_id' I guess you don't
> need it, but maybe we should add it to be complete

I tried adding these from struct perf_sample.id and struct 
perf_sample.sample_id respectively. I then tested these on a perf.data 
which contains a throttle event. The values which my python script 
reported were 0xFFFFFFFFFFFFFFFF -- is this expected? If so, I'll send 
my revised patch.

Thanks,
Stephen

`perf report --header-only` from my testing perf.data:

# ========
# captured on    : Wed Aug 11 11:55:44 2021
# header version : 1
# data offset    : 264
# data size      : 18240
# feat offset    : 18504
# hostname : stepbren-ol7-2
# os release : 5.4.17-2102.203.6.el7uek.x86_64
# perf version : 5.4.17-2102.203.6.el7uek.x86_64
# arch : x86_64
# nrcpus online : 4
# nrcpus avail : 4
# cpudesc : Intel(R) Xeon(R) Platinum 8167M CPU @ 2.00GHz
# cpuid : GenuineIntel,6,85,4
# total memory : 30522624 kB
# cmdline : /usr/libexec/perf.5.4.17-2102.203.6.el7uek.x86_64 record -c 
100000 -e cycles -- sh -c echo 2000 > /proc/irqoff_sleep_millis
# event : name = cycles, , id = { 5, 6, 7, 8 }, size = 112, { 
sample_period, sample_freq } = 100000, sample_type = IP|TID|TIME, 
read_format = ID, disabled = 1, inherit = 1, mmap = 1, comm = 1, 
enable_on_exec = 1, task = 1, sample_id_all = 1, exclude_guest = 1, 
mmap2 = 1, comm_exec = 1, ksymbol = 1, bpf_event = 1
# CPU_TOPOLOGY info available, use -I to display
# NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: software = 1, uprobe = 7, cpu = 4, breakpoint = 5, 
tracepoint = 2, kprobe = 6, msr = 8
# CACHE info available, use -I to display
# time of first sample : 228.321751
# time of last sample : 230.362698
# sample duration :   2040.947 ms
# MEM_TOPOLOGY info available, use -I to display
# missing features: TRACING_DATA BRANCH_STACK GROUP_DESC AUXTRACE STAT 
CLOCKID DIR_FORMAT COMPRESSED CPU_PMU_CAPS CLOCK_DATA HYBRID_TOPOLOGY 
HYBRID_CPU_PMU_CAPS
# ========
#