linux-kernel - [PATCH v1 0/6] Add io_dir to avoid memory overhead from opendir

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20231207050433.1426834-1-irogers@google.com>
Date:   Wed,  6 Dec 2023 21:04:27 -0800
From:   Ian Rogers <irogers@...gle.com>
To:     Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Mark Rutland <mark.rutland@....com>,
        Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
        Jiri Olsa <jolsa@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>,
        Ian Rogers <irogers@...gle.com>,
        Adrian Hunter <adrian.hunter@...el.com>,
        "Masami Hiramatsu (Google)" <mhiramat@...nel.org>,
        Thomas Richter <tmricht@...ux.ibm.com>,
        Quentin Monnet <quentin@...valent.com>,
        Fangrui Song <maskray@...gle.com>,
        Kan Liang <kan.liang@...ux.intel.com>,
        Yanteng Si <siyanteng@...ngson.cn>,
        K Prateek Nayak <kprateek.nayak@....com>,
        Artem Savkov <asavkov@...hat.com>,
        James Clark <james.clark@....com>,
        Ravi Bangoria <ravi.bangoria@....com>,
        linux-kernel@...r.kernel.org, linux-perf-users@...r.kernel.org
Subject: [PATCH v1 0/6] Add io_dir to avoid memory overhead from opendir

glibc's opendir allocates a minimum of 32kb, when called recursively
for a directory tree the memory consumption can add up - nearly 300kb
during perf start-up when processing modules. Add a stack allocated
variant of readdir sized a little more than 1kb

This was previously part of the memory saving change set:
https://lore.kernel.org/lkml/20231127220902.1315692-1-irogers@google.com/
It is separated here and a feature check and syscall workaround for
missing getdents64 added.

Ian Rogers (6):
  tools build: Add a feature test for getdents64
  tools lib api: Add io_dir an allocation free readdir alternative
  perf maps: Switch modules tree walk to io_dir__readdir
  perf pmu: Switch to io_dir__readdir
  perf header: Switch mem topology to io_dir__readdir
  perf events: Remove scandir in thread synthesis

 tools/build/Makefile.feature          |  1 +
 tools/build/feature/Makefile          |  4 ++
 tools/build/feature/test-all.c        |  5 ++
 tools/build/feature/test-getdents64.c | 12 ++++
 tools/lib/api/Makefile                |  2 +-
 tools/lib/api/io_dir.h                | 84 +++++++++++++++++++++++++++
 tools/perf/Makefile.config            |  4 ++
 tools/perf/util/header.c              | 31 +++++-----
 tools/perf/util/machine.c             | 19 +++---
 tools/perf/util/pmu.c                 | 48 ++++++---------
 tools/perf/util/pmus.c                | 30 ++++------
 tools/perf/util/synthetic-events.c    | 22 +++----
 12 files changed, 177 insertions(+), 85 deletions(-)
 create mode 100644 tools/build/feature/test-getdents64.c
 create mode 100644 tools/lib/api/io_dir.h

-- 
2.43.0.rc2.451.g8631bc7472-goog