linux-kernel - Re: [PATCH 2/2] torture: Make thread detection more robust by using lspcu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <bfab82a0-bf21-7782-f6fb-3adaf827ad25@molgen.mpg.de>
Date:   Fri, 25 Feb 2022 06:49:32 +0100
From:   Paul Menzel <pmenzel@...gen.mpg.de>
To:     "Paul E. McKenney" <paulmck@...nel.org>
Cc:     Josh Triplett <josh@...htriplett.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        Lai Jiangshan <jiangshanlai@...il.com>,
        Shuah Khan <shuah@...nel.org>,
        Zhouyi Zhou <zhouzhouyi@...il.com>, rcu@...r.kernel.org,
        linux-kselftest@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 2/2] torture: Make thread detection more robust by using
 lspcu

Dear Paul,


Am 24.02.22 um 21:56 schrieb Paul E. McKenney:
> On Thu, Feb 24, 2022 at 09:24:11AM +0100, Paul Menzel wrote:

>> Am 22.02.22 um 18:43 schrieb Paul E. McKenney:
>>> On Tue, Feb 22, 2022 at 01:07:17PM +0100, Paul Menzel wrote:
>>>> For consecutive numbers *lscpu* collapses the output and just shows the
>>>> range with start and end. The processors are numbered that way on POWER8.
>>>>
>>>>       $ sudo ppc64_cpu --smt=8
>>>>       $ lscpu | grep '^NUMA node'
>>>>       NUMA node(s):                    2
>>>>       NUMA node0 CPU(s):               0-79
>>>>       NUMA node8 CPU(s):               80-159
>>>>
>>>> This causes the heuristic to detect the number threads per core, looking
>>>> for the number after the first comma, to fail, and QEMU aborts because of
>>>> invalid arguments.
>>>>
>>>>       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>>>>       $
>>>>
>>>> (Before the last patch, the whole line was returned.)
>>>>
>>>>       $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
>>>>       NUMA node0 CPU(s):               0-79
>>>>
>>>> *lscpu* shows the number of threads per core, so use that value directly.
>>>>
>>>>       $ sudo ppc64_cpu --smt=8
>>>>       $ lscpu | grep 'Thread(s) per core'
>>>>       Thread(s) per core:              8
>>>>       $ sudo ppc64_cpu --smt=off
>>>>       $ lscpu | grep 'Thread(s) per core'
>>>>       Thread(s) per core:              1
>>>>
>>>> Note, the replaced heuristic is also incorrect for that case, where the
>>>> threads per core are disabled.
>>>>
>>>>       $ lscpu | sed -n -e '/^NUMA node0/s/^[^,]*,\([0-9]*\),.*$/\1/p'
>>>>       8
>>>>
>>>> Signed-off-by: Paul Menzel <pmenzel@...gen.mpg.de>
>>>
>>> Makes sense, and thank you for chasing this down and for the fix!
>>>
>>> But should this patch and 1/2 be merged?  Or am I confused and they
>>> are somehow affecting two different lines of scripting?
>>
>> You are right. I guess with 1/2 I just wanted to document clearly, what I
>> learned in #sed@....libera.chat, that means, how to avoid using grep, when
>> sed is used.
> 
> Nothing wrong with that!
> 
> I have merged the two patches as shown below.  Does this work for you?
> 
> 							Thanx, Paul
> 
> ------------------------------------------------------------------------
> 
> commit 9f0daba62e958c31326c7a9eae33651e3a3cc6b4
> Author: Paul Menzel <pmenzel@...gen.mpg.de>
> Date:   Tue Feb 22 13:07:16 2022 +0100
> 
>      torture: Make thread detection more robust by using lspcu
>      
>      For consecutive numbers the lscpu command collapses the output and just
>      shows the range with start and end. The processors are numbered that
>      way on POWER8.
>      
>          $ sudo ppc64_cpu --smt=8
>          $ lscpu | grep '^NUMA node'
>          NUMA node(s):                    2
>          NUMA node0 CPU(s):               0-79
>          NUMA node8 CPU(s):               80-159
>      
>      This causes the heuristic to detect the number threads per core, looking
>      for the number after the first comma, to fail, and QEMU aborts because of
>      invalid arguments.
>      
>          $ lscpu | grep '^NUMA node0' | sed -e 's/^[^,-]*(,|\-)\([0-9]*\),.*$/\1/'
>          NUMA node0 CPU(s):               0-79
>      
>      But the lscpu command shows the number of threads per core:
>      
>          $ sudo ppc64_cpu --smt=8
>          $ lscpu | grep 'Thread(s) per core'
>          Thread(s) per core:              8
>          $ sudo ppc64_cpu --smt=off
>          $ lscpu | grep 'Thread(s) per core'
>          Thread(s) per core:              1
>      
>      This commit therefore directly uses that value.

Maybe extend: …, and replaces `grep` by using using sed’s switch `-n` 
and the command p.

>      
>      Signed-off-by: Paul Menzel <pmenzel@...gen.mpg.de>
>      Signed-off-by: Paul E. McKenney <paulmck@...nel.org>
> 
> diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
> index c35ba24f994c3..66d0414d8e4bc 100644
> --- a/tools/testing/selftests/rcutorture/bin/functions.sh
> +++ b/tools/testing/selftests/rcutorture/bin/functions.sh
> @@ -301,7 +301,7 @@ specify_qemu_cpus () {
>   			echo $2 -smp $3
>   			;;
>   		qemu-system-ppc64)
> -			nt="`lscpu | grep '^NUMA node0' | sed -e 's/^[^,]*,\([0-9]*\),.*$/\1/'`"
> +			nt="`lscpu | sed -n 's/^Thread(s) per core:\s*//p'`"
>   			echo $2 -smp cores=`expr \( $3 + $nt - 1 \) / $nt`,threads=$nt
>   			;;
>   		esac

Thank you for doing that, and sorry for the extra work.


Kind regards,

Paul