linux-kernel - Re: [PATCH 05/12] selftests/mm: fix invocation of tests that are run via shell scripts

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6f845148-e180-822d-a54f-e2bb844d54f8@nvidia.com>
Date:   Fri, 2 Jun 2023 13:38:21 -0700
From:   John Hubbard <jhubbard@...dia.com>
To:     David Hildenbrand <david@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>
CC:     Peter Xu <peterx@...hat.com>, Shuah Khan <shuah@...nel.org>,
        "Nathan Chancellor" <nathan@...nel.org>, <linux-mm@...ck.org>,
        <linux-kselftest@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 05/12] selftests/mm: fix invocation of tests that are run
 via shell scripts

On 6/2/23 03:05, David Hildenbrand wrote:
> On 02.06.23 03:33, John Hubbard wrote:
>> We cannot depend upon git to reliably retain the executable bit on shell
>> scripts, or so I was told several years ago while working on this same
>> run_vmtests.sh script. And sure enough, things such as test_hmm.sh are
>> lately failing to run, due to lacking execute permissions.
>>
>> A nice clean way to fix this would have been to use TEST_PROGS instead
>> of TEST_FILES for the .sh scripts here. That tells the selftest
>> framework to run these (and emit a warning if the files are not
>> executable, but still run them anyway).

Actually, for the record (and I'll update this in v2), the above is
inaccurate, because run_vmtests.sh aspires to be the only TEST_PROGS
item here. And I see that the framework does already work if-and-only-if
invoked via Make, as in "make run_tests".

However,

a) Many people naturally expect to run test scripts without
(unnecessarily!) involving Make, and

b) Based on some experience in building and using various test
frameworks over many years, I'd claim that it's better to use shell
scripts to collect and manage tests and test scripts, rather than
involving Make. Make is a limited, specialized language and is better at
handling builds and dependencies.

So the "make run_tests" is a convenience, but it should not be the only
way to launch a test run. So we still want to fix this up.

>>
>> Unfortunately, run_vmtests.sh has its own run_test() routine, which does
>> *not* do the right thing for shell scripts.
>>
>> Fix this by explicitly adding "bash" to each of the shell script
>> invocations. Leave fixing the overall approach to another day.
>>
>> Signed-off-by: John Hubbard <jhubbard@...dia.com>
>> ---
>>   tools/testing/selftests/mm/run_vmtests.sh | 6 +++---
>>   1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
>> index 4893eb60d96d..8f81432e4bac 100644
>> --- a/tools/testing/selftests/mm/run_vmtests.sh
>> +++ b/tools/testing/selftests/mm/run_vmtests.sh
>> @@ -242,18 +242,18 @@ if [ $VADDR64 -ne 0 ]; then
>>       if [ "$ARCH" == "$ARCH_ARM64" ]; then
>>           echo 6 > /proc/sys/vm/nr_hugepages
>>       fi
>> -    CATEGORY="hugevm" run_test ./va_high_addr_switch.sh
>> +    CATEGORY="hugevm" run_test bash ./va_high_addr_switch.sh
>>       if [ "$ARCH" == "$ARCH_ARM64" ]; then
>>           echo $prev_nr_hugepages > /proc/sys/vm/nr_hugepages
>>       fi
>>   fi # VADDR64
>>   # vmalloc stability smoke test
>> -CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke
>> +CATEGORY="vmalloc" run_test bash ./test_vmalloc.sh smoke
>>   CATEGORY="mremap" run_test ./mremap_dontunmap
>> -CATEGORY="hmm" run_test ./test_hmm.sh smoke
>> +CATEGORY="hmm" run_test bash ./test_hmm.sh smoke
>>   # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests
>>   CATEGORY="madv_populate" run_test ./madv_populate
> 
> Sounds hacky, but if it gets the job done
> 

Yes. It's also hacky that we can't just invoke shell scripts like normal
programs. This limitation hurts my sense of "things should be more
perfect!". :)

> Acked-by: David Hildenbrand <david@...hat.com>
> 

Thanks for the ack.

-- 
John Hubbard
NVIDIA