| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Fri, 25 Feb 2022 12:45:25 -0800
From: "Paul E. McKenney" <paulmck@...nel.org>
To: Paul Menzel <pmenzel@...gen.mpg.de>
Cc: Josh Triplett <josh@...htriplett.org>,
Steven Rostedt <rostedt@...dmis.org>,
Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
Lai Jiangshan <jiangshanlai@...il.com>,
Shuah Khan <shuah@...nel.org>,
Zhouyi Zhou <zhouzhouyi@...il.com>, rcu@...r.kernel.org,
linux-kselftest@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 2/2] torture: Make thread detection more robust by using
lspcu
On Fri, Feb 25, 2022 at 06:49:32AM +0100, Paul Menzel wrote:
> Dear Paul,
>
>
> Am 24.02.22 um 21:56 schrieb Paul E. McKenney:
> > On Thu, Feb 24, 2022 at 09:24:11AM +0100, Paul Menzel wrote:
>
> > > Am 22.02.22 um 18:43 schrieb Paul E. McKenney:
> > > > On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
> > > > > For consecutive numbers *lscpu* collapses the output and just shows the
> > > > > range with start and end. The processors are numbered that way on POWER8.
> > > > >
> > > > > $ sudo ppc64_cpu --smt=8
> > > > > $ lscpu | grep '^NUMA node'
> > > > > NUMA node(s): 2
> > > > > NUMA node0 CPU(s): 0-79
> > > > > NUMA node8 CPU(s): 80-159
> > > > >
> > > > > This causes the heuristic to detect the number threads per core, looking
> > > > > for the number after the first comma, to fail, and QEMU aborts because of
> > > > > invalid arguments.
> > > > >
> > > > > $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > > > > $
> > > > >
> > > > > (Before the last patch, the whole line was returned.)
> > > > >
> > > > > $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
> > > > > NUMA node0 CPU(s): 0-79
> > > > >
> > > > > *lscpu* shows the number of threads per core, so use that value directly.
> > > > >
> > > > > $ sudo ppc64_cpu --smt=8
> > > > > $ lscpu | grep 'Thread(s) per core'
> > > > > Thread(s) per core: 8
> > > > > $ sudo ppc64_cpu --smt=off
> > > > > $ lscpu | grep 'Thread(s) per core'
> > > > > Thread(s) per core: 1
> > > > >
> > > > > Note, the replaced heuristic is also incorrect for that case, where the
> > > > > threads per core are disabled.
> > > > >
> > > > > $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
> > > > > 8
> > > > >
> > > > > Signed-off-by: Paul Menzel <pmenzel@...gen.mpg.de>
> > > >
> > > > Makes sense, and thank you for chasing this down and for the fix!
> > > >
> > > > But should this patch and 1/2 be merged? Or am I confused and they
> > > > are somehow affecting two different lines of scripting?
> > >
> > > You are right. I guess with 1/2 I just wanted to document clearly, what I
> > > learned in #sed@....libera.chat, that means, how to avoid using grep, when
> > > sed is used.
> >
> > Nothing wrong with that!
> >
> > I have merged the two patches as shown below. Does this work for you?
> >
> > Thanx, Paul
> >
> > ------------------------------------------------------------------------
> >
> > commit 9f0daba62e958c31326c7a9eae33651e3a3cc6b4
> > Author: Paul Menzel <pmenzel@...gen.mpg.de>
> > Date: Tue Feb 22 13:07:16 2022 +0100
> >
> > torture: Make thread detection more robust by using lspcu
> > For consecutive numbers the lscpu command collapses the output and just
> > shows the range with start and end. The processors are numbered that
> > way on POWER8.
> > $ sudo ppc64_cpu --smt=8
> > $ lscpu | grep '^NUMA node'
> > NUMA node(s): 2
> > NUMA node0 CPU(s): 0-79
> > NUMA node8 CPU(s): 80-159
> > This causes the heuristic to detect the number threads per core, looking
> > for the number after the first comma, to fail, and QEMU aborts because of
> > invalid arguments.
> > $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
> > NUMA node0 CPU(s): 0-79
> > But the lscpu command shows the number of threads per core:
> > $ sudo ppc64_cpu --smt=8
> > $ lscpu | grep 'Thread(s) per core'
> > Thread(s) per core: 8
> > $ sudo ppc64_cpu --smt=off
> > $ lscpu | grep 'Thread(s) per core'
> > Thread(s) per core: 1
> > This commit therefore directly uses that value.
>
> Maybe extend: …, and replaces `grep` by using using sed’s switch `-n` and
> the command p.
Ah, good point -- I will update this on the next rebase.
> > Signed-off-by: Paul Menzel <pmenzel@...gen.mpg.de>
> > Signed-off-by: Paul E. McKenney <paulmck@...nel.org>
> >
> > diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
> > index c35ba24f994c3..66d0414d8e4bc 100644
> > --- a/tools/testing/selftests/rcutorture/bin/functions.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/functions.sh
> > @@ -301,7 +301,7 @@ specify_qemu_cpus () {
> > echo $2 -smp $3
> > ;;
> > qemu-system-ppc64)
> > - nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
> > + nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
> > echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
> > ;;
> > esac
>
> Thank you for doing that, and sorry for the extra work.
No complaints! You are after all testing on powerpc, and getting that
working better is a good thing.
Thanx, Paul
Powered by blists - more mailing lists