linux-kernel - Re: [PATCH v1 9/9] selftests/mm: Run all tests from run

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <a8084196-38fc-7bbe-174c-3db4c174e3c8@arm.com>
Date:   Thu, 13 Jul 2023 16:30:18 +0100
From:   Ryan Roberts <ryan.roberts@....com>
To:     David Hildenbrand <david@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Shuah Khan <shuah@...nel.org>,
        Jérôme Glisse <jglisse@...hat.com>,
        Mark Brown <broonie@...nel.org>,
        John Hubbard <jhubbard@...dia.com>,
        Florent Revest <revest@...omium.org>,
        "Liam R. Howlett" <Liam.Howlett@...cle.com>
Cc:     linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        linux-kselftest@...r.kernel.org, Peter Xu <peterx@...hat.com>
Subject: Re: [PATCH v1 9/9] selftests/mm: Run all tests from run_vmtests.sh

On 13/07/2023 16:25, David Hildenbrand wrote:
>>>
>>> Which run_kselftest.sh are you referring to, the one in the parent directory?
>>
>> run_kselftest.sh is the uniform way of executing all the kselftests. mm seems to
>> be trying to be special as far as I can see. Certainly if you run the `install`
>> make target, kselftests will create a list of all the tests (including non-mm
>> tests if you have included them in the TARGETS variable) and copy that test list
>> and run_kselftest.sh to the install path along with all the test binaries. Then
>> the user can invoke any of the collections or specific tests in the collections
>> using that tool. It also wraps everything with tap output, runs tests with a
>> timeout, etc.
>>
>> See Documentation/dev-tools/kselftest.rst
>>
> 
> Got it, thanks!
> 
>>>
>>> How to invoke it to run these mm tests?
>>>
>>> (I never dared invoking something different than
>>> run_vmtests.sh ;) )
>>
>> # single test:
>> $ sudo ./run_kselftest.sh -t mm:<test_name>
>>
>> or
>>
>> # all tests in collection:
>> $ sudo ./run_kselftest.sh -c mm
>>
> 
> Ah, that makes sense. So I guess mm is then one "collection".

yep, the collections are the directories under tools/testing/selftests with a
few special exceptions.

> 
>>>
>>> [...]
>>>
>>>>
>>>> Signed-off-by: Ryan Roberts <ryan.roberts@....com>
>>>> ---
>>>>    tools/testing/selftests/mm/Makefile       | 79 ++++++++++++-----------
>>>>    tools/testing/selftests/mm/run_vmtests.sh | 23 +++++++
>>>>    tools/testing/selftests/mm/settings       |  2 +-
>>>>    3 files changed, 64 insertions(+), 40 deletions(-)
>>>>
>>>> diff --git a/tools/testing/selftests/mm/Makefile
>>>> b/tools/testing/selftests/mm/Makefile
>>>> index 66d7c07dc177..881ed96d96fd 100644
>>>> --- a/tools/testing/selftests/mm/Makefile
>>>> +++ b/tools/testing/selftests/mm/Makefile
>>>> @@ -35,39 +35,39 @@ MAKEFLAGS += --no-builtin-rules
>>>>    CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
>>>>    LDLIBS = -lrt -lpthread
>>>>    -TEST_GEN_PROGS = cow
>>>> -TEST_GEN_PROGS += compaction_test
>>>> -TEST_GEN_PROGS += gup_longterm
>>>> -TEST_GEN_PROGS += gup_test
>>>> -TEST_GEN_PROGS += hmm-tests
>>>> -TEST_GEN_PROGS += hugetlb-madvise
>>>> -TEST_GEN_PROGS += hugepage-mmap
>>>> -TEST_GEN_PROGS += hugepage-mremap
>>>> -TEST_GEN_PROGS += hugepage-shm
>>>> -TEST_GEN_PROGS += hugepage-vmemmap
>>>> -TEST_GEN_PROGS += khugepaged
>>>> -TEST_GEN_PROGS += madv_populate
>>>> -TEST_GEN_PROGS += map_fixed_noreplace
>>>> -TEST_GEN_PROGS += map_hugetlb
>>>> -TEST_GEN_PROGS += map_populate
>>>> -TEST_GEN_PROGS += memfd_secret
>>>> -TEST_GEN_PROGS += migration
>>>> -TEST_GEN_PROGS += mkdirty
>>>> -TEST_GEN_PROGS += mlock-random-test
>>>> -TEST_GEN_PROGS += mlock2-tests
>>>> -TEST_GEN_PROGS += mrelease_test
>>>> -TEST_GEN_PROGS += mremap_dontunmap
>>>> -TEST_GEN_PROGS += mremap_test
>>>> -TEST_GEN_PROGS += on-fault-limit
>>>> -TEST_GEN_PROGS += thuge-gen
>>>> -TEST_GEN_PROGS += transhuge-stress
>>>> -TEST_GEN_PROGS += uffd-stress
>>>> -TEST_GEN_PROGS += uffd-unit-tests
>>>> -TEST_GEN_PROGS += soft-dirty
>>>> -TEST_GEN_PROGS += split_huge_page_test
>>>> -TEST_GEN_PROGS += ksm_tests
>>>> -TEST_GEN_PROGS += ksm_functional_tests
>>>> -TEST_GEN_PROGS += mdwe_test
>>>> +TEST_GEN_FILES = cow
>>>> +TEST_GEN_FILES += compaction_test
>>>> +TEST_GEN_FILES += gup_longterm
>>>> +TEST_GEN_FILES += gup_test
>>>> +TEST_GEN_FILES += hmm-tests
>>>> +TEST_GEN_FILES += hugetlb-madvise
>>>> +TEST_GEN_FILES += hugepage-mmap
>>>> +TEST_GEN_FILES += hugepage-mremap
>>>> +TEST_GEN_FILES += hugepage-shm
>>>> +TEST_GEN_FILES += hugepage-vmemmap
>>>> +TEST_GEN_FILES += khugepaged
>>>> +TEST_GEN_FILES += madv_populate
>>>> +TEST_GEN_FILES += map_fixed_noreplace
>>>> +TEST_GEN_FILES += map_hugetlb
>>>> +TEST_GEN_FILES += map_populate
>>>> +TEST_GEN_FILES += memfd_secret
>>>> +TEST_GEN_FILES += migration
>>>> +TEST_GEN_FILES += mkdirty
>>>> +TEST_GEN_FILES += mlock-random-test
>>>> +TEST_GEN_FILES += mlock2-tests
>>>> +TEST_GEN_FILES += mrelease_test
>>>> +TEST_GEN_FILES += mremap_dontunmap
>>>> +TEST_GEN_FILES += mremap_test
>>>> +TEST_GEN_FILES += on-fault-limit
>>>> +TEST_GEN_FILES += thuge-gen
>>>> +TEST_GEN_FILES += transhuge-stress
>>>> +TEST_GEN_FILES += uffd-stress
>>>> +TEST_GEN_FILES += uffd-unit-tests
>>>> +TEST_GEN_FILES += soft-dirty
>>>> +TEST_GEN_FILES += split_huge_page_test
>>>> +TEST_GEN_FILES += ksm_tests
>>>> +TEST_GEN_FILES += ksm_functional_tests
>>>> +TEST_GEN_FILES += mdwe_test
>>>
>>> IIRC, we recently converted all to TEST_GEN_PROGS. See
>>>
>>> commit aef6fde75d8c6c1cad4a0e017a8d4cbee2143723
>>> Author: Peter Xu <peterx@...hat.com>
>>> Date:   Wed Apr 12 12:42:18 2023 -0400
>>>
>>>      selftests/mm: use TEST_GEN_PROGS where proper
>>>          TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the
>>> mm/Makefile to
>>>      specify programs that need to build.  Logically all these binaries should
>>>      all fall into TEST_GEN_PROGS.
>>>          Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can
>>> reference
>>>      all the tests easily later.
>>>
>>>
>>> Why is that change required, and how does it interact with
>>> run_kselftest.sh? (Not clear from you patch description.)
>>
>> TEST_GEN_PROGS will compile and install the tests and will add them to the list
>> of tests that run_kselftest.sh will run. TEST_GEN_FILES will compile and install
>> the tests but will not add them to the test list.
>>
>> Note that run_vmtests.sh is added to TEST_PROGS, which means it ends up in the
>> test list. (the lack of "_GEN" means it won't be compiled, but simply copied).
>>
>> So with this change at the kselftest level, there is a single test in its list;
>> run_vmtests.sh. And all the other tests that were previously in that list are
>> moved into run_vmtests.sh (if they weren't there already).
> 
> That sound good to me. (worth adding to the patch description)
> 
> Let me CC Peter, so he's aware.

Thanks - would be good to hear his opinion!