linux-kernel - Re: [RFC v1 0/3] C and Rust support for perf script

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <ZvSktIhEOtnmrvUR@google.com>
Date: Wed, 25 Sep 2024 17:03:00 -0700
From: Namhyung Kim <namhyung@...nel.org>
To: StefanEne@...gle.com
Cc: peterz@...radead.org, mingo@...hat.com, acme@...nel.org,
	mark.rutland@....com, alexander.shishkin@...ux.intel.com,
	jolsa@...nel.org, irogers@...gle.com, adrian.hunter@...el.com,
	kan.liang@...ux.intel.com, linux-perf-users@...r.kernel.org,
	linux-kernel@...r.kernel.org, vinicius.gomes@...el.com,
	stefan.ene@...el.com, stef_an_ene@...look.com
Subject: Re: [RFC v1 0/3] C and Rust support for perf script

Hello,

On Thu, Sep 19, 2024 at 02:51:01PM -0700, StefanEne wrote:
> From: Stefan Ene <stefan.ene@...el.com>
> 
> ============================COVER=LETTER============================
> 
> This proposal is addressing the usability and performance of the available scripting languages for custom perf event data processing inside of the perf toolset, specifically with the perf script command.
> 
> With the perf-script custom event processing functionality for C and Rust, we noticed between 2x to 5x speed improvement with our new methods compared to the existent Python and Perl scripting methods.
> 
> To explain the proposed method, you begin with the C or Rust script we’ve templatized, then just add changes for custom event processing in the preferred language using this provided template, compile the respective script into a dynamic library, then give the resulting .so file as a parameter to the newly implemented perf script option.
> 
> List of functionality additions:
> 
> 1/ Added new "--new_script" option inside the perf script command that takes in as parameter .so files. Code addition in tools/perf/builtin-script.c
> 
> 2/ The functional code for the newly implemented option was added to the tools/perf/util/new_script.c and tools/perf/util/new_script.h files.
> 
> 3/ Folder added at tools/perf/new_script_templates, containing C and Rust script templates for the new option, along with compilation instructions.
> 
> 4/ Finally, a short bash script for updating the perf tool set within the kernel code was provided in the base-level file update_perf_tools.sh.
> 
> 
> Common Questions
> 
> * How can I use the new toolset?
> 
>    The new implementations for the perf script have a detailed usage guide inside of the tools/perf/new_script_templates/README file, along with some script templates for C and Rust!
> 
> * Why a new option instead of expanding dlfilter?
> 
>    The new option gave us the flexibility to make use of the fast dlfilter dynamic library approach, as opposed to implementing another interpreting methodology. This allows for scalability, with great potential to other languages supporting dynamic library calls from the base C code.
> 
> * Why use C and Rust instead?
> 
>    As of kernel version 6.11, the perf tool has a large overhead for data processing using Python and Perl, given the languages having to use their respective perf built-in interpreters. Furthermore, while Python is widley used in the development comunity, as of 20204, Perl is only used by 2.5 of developers worldwide, while C and Rust are more common, with 20.3% and 12.6% usage, respectively (Source: statista.com).
> 
> * What are the actual performance improvements?
> 
>    As last tested, the C and Rust approach are anywhere between 2 to 5 times faster than the existent Python and Perl scripting methods, with Rust being the fastest all across!
> 
> 
> Acknowledgements:
> 
> This code was completed as part of an Intel summer internship project, under the mentoring of Vinicius Gomes, Intel Linux Kernel Team.
> 
> =========================END=COVER=LETTER===========================
> 
> 
> Stefan Ene (3):
>   add the new perf script option (--new_script) and related changes
>   added the C sample script
>   added the Rust sample script

Thanks for sharing your work.  But I think there are many more work to
support scripting of native languages properly.  For example, you need
to resolve symbols and callchains before passing them to user.  And we
might think about the safety as it can modify the internal state or
something.

Thanks,
Namhyung

> 
>  tools/perf/builtin-script.c              |  22 +-
>  tools/perf/new_script_templates/README   |  65 ++++
>  tools/perf/new_script_templates/lib.rs   | 108 +++++++
>  tools/perf/new_script_templates/script.c | 113 +++++++
>  tools/perf/util/Build                    |   1 +
>  tools/perf/util/new_script.c             | 376 +++++++++++++++++++++++
>  tools/perf/util/new_script.h             |  54 ++++
>  tools/perf/util/new_script_rs_lib.h      |  35 +++
>  8 files changed, 773 insertions(+), 1 deletion(-)
>  create mode 100644 tools/perf/new_script_templates/README
>  create mode 100644 tools/perf/new_script_templates/lib.rs
>  create mode 100644 tools/perf/new_script_templates/script.c
>  create mode 100644 tools/perf/util/new_script.c
>  create mode 100644 tools/perf/util/new_script.h
>  create mode 100644 tools/perf/util/new_script_rs_lib.h
> 
> -- 
> 2.46.0
>