linux-kernel - [PATCH 0/8] rtla/timerlat: Support actions on threshold and on end

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250611135644.219127-1-tglozar@redhat.com>
Date: Wed, 11 Jun 2025 15:56:36 +0200
From: Tomas Glozar <tglozar@...hat.com>
To: Steven Rostedt <rostedt@...dmis.org>
Cc: linux-trace-kernel@...r.kernel.org,
	linux-kernel@...r.kernel.org,
	John Kacur <jkacur@...hat.com>,
	Luis Goncalves <lgoncalv@...hat.com>,
	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Chang Yin <cyin@...hat.com>,
	Costa Shulyupin <costa.shul@...hat.com>,
	Tomas Glozar <tglozar@...hat.com>
Subject: [PATCH 0/8] rtla/timerlat: Support actions on threshold and on end

This series adds a feature that allows to user to specify certain
kinds of "actions" to be executed at one of two places in the rtla
measurement: when tracing is stopped on latency threshold, and at the
end of tracing.

Two new options are added: -A/--on-threshold, and -N/--on-end, taking
the action as an argument. For example:

$ rtla timerlat hist -T 10 -A shell,command="echo Threshold" \
-N shell,command="echo Tracing stopped"

will print "Threshold" if a thread latency higher than 10 microseconds
is reached, and "Tracing stopped" always at the end.

The list of possible actions is extensible and is covered in
the commit messages. Later, a documentation patch series will be sent
with clear explanation of every action and its syntax.

Notably, a special action "continue" resumes tracing. For example:

$ rtla timerlat hist -T 100 -A shell,command="echo Threshold" \
-A continue -d 10s

will print "Threshold" as many times as tracing is stopped after
thread latency reaches 100us.

The feature was inspired by a case where collecting perf data on rtla
latency overflow was required, which can be done by sending a signal
to the perf process.

Example of this with Intel PT aux buffer:

$ perf record --cpu 0 -e intel_pt// -S -- rtla timerlat top -q -T 100 \
-c 0 -A signal,pid=parent,num=12 -A continue

In general, the feature is aiming to allow integration with external
tooling. To implement even more flexibility, passing context to the
shell through environmental variables, or even an entire scripting
language with access to the rtla internals can be implemented if
needed.

Tomas Glozar (8):
  rtla/timerlat: Introduce enum timerlat_tracing_mode
  rtla/timerlat: Add action on threshold feature
  rtla/timerlat_bpf: Allow resuming tracing
  rtla/timerlat: Add continue action
  rtla/timerlat: Add action on end feature
  rtla/tests: Check rtla output with grep
  rtla/tests: Add tests for actions
  rtla/tests: Limit duration to maximum of 10s

 tools/tracing/rtla/src/Build           |   1 +
 tools/tracing/rtla/src/actions.c       | 260 +++++++++++++++++++++++++
 tools/tracing/rtla/src/actions.h       |  52 +++++
 tools/tracing/rtla/src/timerlat.bpf.c  |  13 +-
 tools/tracing/rtla/src/timerlat.c      |  24 ++-
 tools/tracing/rtla/src/timerlat.h      |  24 ++-
 tools/tracing/rtla/src/timerlat_bpf.c  |  13 ++
 tools/tracing/rtla/src/timerlat_bpf.h  |   3 +
 tools/tracing/rtla/src/timerlat_hist.c | 145 ++++++++++----
 tools/tracing/rtla/src/timerlat_top.c  | 167 ++++++++++------
 tools/tracing/rtla/tests/engine.sh     |  21 +-
 tools/tracing/rtla/tests/hwnoise.t     |   8 +-
 tools/tracing/rtla/tests/osnoise.t     |   4 +-
 tools/tracing/rtla/tests/timerlat.t    |  36 +++-
 14 files changed, 652 insertions(+), 119 deletions(-)
 create mode 100644 tools/tracing/rtla/src/actions.c
 create mode 100644 tools/tracing/rtla/src/actions.h

-- 
2.49.0