linux-kernel - Re: [PATCH v2] perf test record.sh: Raise limit of open file descriptors

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <ZroMXJoNXCsh2ghd@x1>
Date: Mon, 12 Aug 2024 10:21:32 -0300
From: Arnaldo Carvalho de Melo <acme@...nel.org>
To: vmolnaro@...hat.com
Cc: Michael Petlan <mpetlan@...hat.com>, linux-perf-users@...r.kernel.org,
	acme@...hat.com, Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...hat.com>, Namhyung Kim <namhyung@...nel.org>,
	Mark Rutland <mark.rutland@....com>,
	Alexander Shishkin <alexander.shishkin@...ux.intel.com>,
	Jiri Olsa <jolsa@...nel.org>, Ian Rogers <irogers@...gle.com>,
	Adrian Hunter <adrian.hunter@...el.com>,
	"Liang, Kan" <kan.liang@...ux.intel.com>,
	Athira Rajeev <atrajeev@...ux.vnet.ibm.com>,
	Veronika Molnarova <vmolnaro@...hat.com>,
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2] perf test record.sh: Raise limit of open file
 descriptors

On Mon, Aug 12, 2024 at 10:16:10AM -0300, Arnaldo Carvalho de Melo wrote:
> On Tue, May 21, 2024 at 11:27:56AM +0200, Michael Petlan wrote:
> > On Mon, 29 Apr 2024, vmolnaro@...hat.com wrote:
> > > From: Veronika Molnarova <vmolnaro@...hat.com>
> > > 
> > > Subtest for system-wide record with '--threads=cpu' option fails due
> > > to a limit of open file descriptors on systems with 128 or more CPUs
> > > as the default limit is set to 1024.
> > > 
> > > The number of open file descriptors should be slightly above
> > > nmb_events*nmb_cpus + nmb_cpus(for perf.data.n) + 4*nmb_cpus(for pipes),
> > > which equals 8*nmb_cpus. Therefore, temporarily raise the limit to
> > > 16*nmb_cpus for the test.
> > > 
> > > Signed-off-by: Veronika Molnarova <vmolnaro@...hat.com>
> > 
> > Acked-by: Michael Petlan <mpetlan@...hat.com>
> 
> b4 got confused, I'll pick the rigth version manually...

And you forgot to add the perf tools reviewers, as listed in
MAINTAINERS, you can get the list of people who should be on the Cc
using:

⬢[acme@...lbox perf-tools-next]$ scripts/get_maintainer.pl tools/perf/tests/shell/record.sh
Peter Zijlstra <peterz@...radead.org> (supporter:PERFORMANCE EVENTS SUBSYSTEM)
Ingo Molnar <mingo@...hat.com> (supporter:PERFORMANCE EVENTS SUBSYSTEM)
Arnaldo Carvalho de Melo <acme@...nel.org> (supporter:PERFORMANCE EVENTS SUBSYSTEM,commit_signer:3/4=75%)
Namhyung Kim <namhyung@...nel.org> (supporter:PERFORMANCE EVENTS SUBSYSTEM,commit_signer:1/4=25%)
Mark Rutland <mark.rutland@....com> (reviewer:PERFORMANCE EVENTS SUBSYSTEM)
Alexander Shishkin <alexander.shishkin@...ux.intel.com> (reviewer:PERFORMANCE EVENTS SUBSYSTEM)
Jiri Olsa <jolsa@...nel.org> (reviewer:PERFORMANCE EVENTS SUBSYSTEM)
Ian Rogers <irogers@...gle.com> (reviewer:PERFORMANCE EVENTS SUBSYSTEM)
Adrian Hunter <adrian.hunter@...el.com> (reviewer:PERFORMANCE EVENTS SUBSYSTEM,commit_signer:1/4=25%,authored:1/4=25%,added_lines:7/55=13%,removed_lines:1/1=100%)
"Liang, Kan" <kan.liang@...ux.intel.com> (reviewer:PERFORMANCE EVENTS SUBSYSTEM,commit_signer:1/4=25%,authored:1/4=25%,added_lines:30/55=55%)
Athira Rajeev <atrajeev@...ux.vnet.ibm.com> (commit_signer:1/4=25%,authored:1/4=25%)
Veronika Molnarova <vmolnaro@...hat.com> (authored:1/4=25%,added_lines:17/55=31%)
linux-perf-users@...r.kernel.org (open list:PERFORMANCE EVENTS SUBSYSTEM)
linux-kernel@...r.kernel.org (open list:PERFORMANCE EVENTS SUBSYSTEM)
⬢[acme@...lbox perf-tools-next]$


 
> ⬢[acme@...lbox perf-tools-next]$ b4 am -ctsl --cc-trailers 20240429085721.10122-1-vmolnaro@...hat.com
> Grabbing thread from lore.kernel.org/all/20240429085721.10122-1-vmolnaro@...hat.com/t.mbox.gz
> Checking for newer revisions
> Grabbing search results from lore.kernel.org
> Analyzing 7 messages in the thread
> WARNING: duplicate messages found at index 1
>    Subject 1: perf test record.sh: Raise limit of open file descriptors
>    Subject 2: perf test record.sh: Raise limit of open file descriptors
>   2 is not a reply... assume additional patch
> Looking for additional code-review trailers on lore.kernel.org
> Will use the latest revision: v2
> You can pick other revisions using the -vN flag
> Checking attestation on all messages, may take a moment...
> ---
>   ✓ [PATCH v2] perf test record.sh: Raise limit of open file descriptors
>     + Link: https://lore.kernel.org/r/20240311081127.7652-1-vmolnaro@redhat.com (✓ DKIM/kernel.org)
>     + Acked-by: Michael Petlan <mpetlan@...hat.com> (✓ DKIM/redhat.com)
>     + Signed-off-by: Arnaldo Carvalho de Melo <acme@...hat.com>
>   ✓ [PATCH v2] perf test record.sh: Raise limit of open file descriptors
>     + Acked-by: Michael Petlan <mpetlan@...hat.com> (✓ DKIM/redhat.com)
>     + Link: https://lore.kernel.org/r/20240429085721.10122-1-vmolnaro@redhat.com
>     + Signed-off-by: Arnaldo Carvalho de Melo <acme@...hat.com>
>   ---
>   ✓ Signed: DKIM/redhat.com
> ---
> Total patches: 2
> ---
>  Link: https://lore.kernel.org/r/20240311081127.7652-1-vmolnaro@redhat.com
>  Base: not specified
>        git am ./v2_20240311_vmolnaro_perf_test_record_sh_raise_limit_of_open_file_descriptors.mbx
> ⬢[acme@...lbox perf-tools-next]$
> 
> > > ---
> > > Reworked the patch as the testfile has already changed since the patch
> > > was sent causing the applying to fail, now applying upstream without
> > > problems.
> > >  tools/perf/tests/shell/record.sh | 17 +++++++++++++++++
> > >  1 file changed, 17 insertions(+)
> > > 
> > > diff --git a/tools/perf/tests/shell/record.sh b/tools/perf/tests/shell/record.sh
> > > index 3d1a7759a7b2..344c0e40ebe8 100755
> > > --- a/tools/perf/tests/shell/record.sh
> > > +++ b/tools/perf/tests/shell/record.sh
> > > @@ -21,6 +21,15 @@ testprog="perf test -w thloop"
> > >  cpu_pmu_dir="/sys/bus/event_source/devices/cpu*"
> > >  br_cntr_file="/caps/branch_counter_nr"
> > >  br_cntr_output="branch stack counters"
> > > +default_fd_limit=$(ulimit -n)
> > > +# With option --threads=cpu the number of open file descriptors should be
> > > +# equal to sum of:    nmb_cpus * nmb_events (2+dummy),
> > > +#                     nmb_threads for perf.data.n (equal to nmb_cpus) and
> > > +#                     2*nmb_cpus of pipes = 4*nmb_cpus (each pipe has 2 ends)
> > > +# All together it needs 8*nmb_cpus file descriptors plus some are also used
> > > +# outside of testing, thus raising the limit to 16*nmb_cpus
> > > +min_fd_limit=$(($(getconf _NPROCESSORS_ONLN) * 16))
> > > +
> > >  
> > >  cleanup() {
> > >    rm -rf "${perfdata}"
> > > @@ -190,11 +199,19 @@ test_branch_counter() {
> > >    echo "Basic branch counter test [Success]"
> > >  }
> > >  
> > > +# raise the limit of file descriptors to minimum
> > > +if [[ $default_fd_limit -lt $min_fd_limit ]]; then
> > > +       ulimit -n $min_fd_limit
> > > +fi
> > > +
> > >  test_per_thread
> > >  test_register_capture
> > >  test_system_wide
> > >  test_workload
> > >  test_branch_counter
> > >  
> > > +# restore the default value
> > > +ulimit -n $default_fd_limit
> > > +
> > >  cleanup
> > >  exit $err
> > > -- 
> > > 2.43.0
> > > 
> > > 
> >