linux-kernel - Re: [PATCH v2 0/7] Add io_dir to avoid memory overhead from opendir

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Z7ZTAR5cN3mqEq8L@google.com>
Date: Wed, 19 Feb 2025 13:54:09 -0800
From: Namhyung Kim <namhyung@...nel.org>
To: Ian Rogers <irogers@...gle.com>
Cc: Peter Zijlstra <peterz@...radead.org>, Ingo Molnar <mingo@...hat.com>,
	Arnaldo Carvalho de Melo <acme@...nel.org>,
	Mark Rutland <mark.rutland@....com>,
	Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
	Jiri Olsa <jolsa@...nel.org>,
	Adrian Hunter <adrian.hunter@...el.com>,
	Kan Liang <kan.liang@...ux.intel.com>, Sam James <sam@...too.org>,
	Jesper Juhl <jesperjuhl76@...il.com>,
	James Clark <james.clark@...aro.org>,
	Zhongqiu Han <quic_zhonhan@...cinc.com>,
	Yicong Yang <yangyicong@...ilicon.com>,
	Thomas Richter <tmricht@...ux.ibm.com>,
	Michael Petlan <mpetlan@...hat.com>,
	Veronika Molnarova <vmolnaro@...hat.com>,
	Anne Macedo <retpolanne@...teo.net>,
	Dominique Martinet <asmadeus@...ewreck.org>,
	Jean-Philippe Romain <jean-philippe.romain@...s.st.com>,
	Junhao He <hejunhao3@...wei.com>, linux-kernel@...r.kernel.org,
	linux-perf-users@...r.kernel.org,
	Krzysztof Łopatowski <krzysztof.m.lopatowski@...il.com>
Subject: Re: [PATCH v2 0/7] Add io_dir to avoid memory overhead from opendir

On Fri, Feb 07, 2025 at 03:24:41PM -0800, Ian Rogers wrote:
> glibc's opendir allocates a minimum of 32kb, when called recursively
> for a directory tree the memory consumption can add up - nearly 300kb
> during perf start-up when processing modules. Add a stack allocated
> variant of readdir sized a little more than 1kb
> 
> v2: Remove the feature test and always use a perf supplied getdents64
>     to workaround an Alpine Linux issue in v1:
>     https://lore.kernel.org/lkml/20231207050433.1426834-1-irogers@google.com/
>     As suggested by Krzysztof Łopatowski
>     <krzysztof.m.lopatowski@...il.com> who also pointed to the perf
>     trace performance improvements in start-up time eliminating stat
>     calls can achieve:
>     https://lore.kernel.org/lkml/20250206113314.335376-2-krzysztof.m.lopatowski@gmail.com/

Let me pick up Krzysztof's patch first.

Thanks,
Namhyung


>     Convert parse-events and hwmon_pmu to use io_dir.
> v1: This was previously part of the memory saving change set:
>     https://lore.kernel.org/lkml/20231127220902.1315692-1-irogers@google.com/
>     It is separated here and a feature check and syscall workaround
>     for missing getdents64 added.
> 
> Ian Rogers (7):
>   tools lib api: Add io_dir an allocation free readdir alternative
>   perf maps: Switch modules tree walk to io_dir__readdir
>   perf pmu: Switch to io_dir__readdir
>   perf header: Switch mem topology to io_dir__readdir
>   perf events: Remove scandir in thread synthesis
>   perf parse-events: Switch tracepoints to io_dir__readdir
>   perf hwmon_pmu: Switch event discovery to io_dir__readdir
> 
>  tools/lib/api/Makefile             |  2 +-
>  tools/lib/api/io_dir.h             | 91 ++++++++++++++++++++++++++++++
>  tools/perf/util/header.c           | 31 +++++-----
>  tools/perf/util/hwmon_pmu.c        | 42 ++++++--------
>  tools/perf/util/machine.c          | 19 +++----
>  tools/perf/util/parse-events.c     | 32 ++++++-----
>  tools/perf/util/pmu.c              | 46 +++++++--------
>  tools/perf/util/pmus.c             | 30 ++++------
>  tools/perf/util/synthetic-events.c | 22 ++++----
>  9 files changed, 194 insertions(+), 121 deletions(-)
>  create mode 100644 tools/lib/api/io_dir.h
> 
> -- 
> 2.48.1.502.g6dc24dfdaf-goog
>