[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <0e512516-6453-f7cb-67ee-f32c17eceea2@arm.com>
Date: Thu, 20 Jul 2023 09:14:17 +0100
From: Ryan Roberts <ryan.roberts@....com>
To: Peter Xu <peterx@...hat.com>, David Hildenbrand <david@...hat.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>,
Shuah Khan <shuah@...nel.org>,
Jérôme Glisse <jglisse@...hat.com>,
Mark Brown <broonie@...nel.org>,
John Hubbard <jhubbard@...dia.com>,
Florent Revest <revest@...omium.org>,
linux-kernel@...r.kernel.org, linux-mm@...ck.org,
linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v2 8/8] selftests/mm: Run all tests from run_vmtests.sh
On 19/07/2023 21:45, Peter Xu wrote:
> On Mon, Jul 17, 2023 at 07:27:13PM +0200, David Hildenbrand wrote:
>> On 17.07.23 12:31, Ryan Roberts wrote:
>>> It is very unclear to me how one is supposed to run all the mm selftests
>>> consistently and get clear results.
>>>
>>> Most of the test programs are launched by both run_vmtests.sh and
>>> run_kselftest.sh:
>>>
>>> hugepage-mmap
>>> hugepage-shm
>>> map_hugetlb
>>> hugepage-mremap
>>> hugepage-vmemmap
>>> hugetlb-madvise
>>> map_fixed_noreplace
>>> gup_test
>>> gup_longterm
>>> uffd-unit-tests
>>> uffd-stress
>>> compaction_test
>>> on-fault-limit
>>> map_populate
>>> mlock-random-test
>>> mlock2-tests
>>> mrelease_test
>>> mremap_test
>>> thuge-gen
>>> virtual_address_range
>>> va_high_addr_switch
>>> mremap_dontunmap
>>> hmm-tests
>>> madv_populate
>>> memfd_secret
>>> ksm_tests
>>> ksm_functional_tests
>>> soft-dirty
>>> cow
>>>
>>> However, of this set, when launched by run_vmtests.sh, some of the
>>> programs are invoked multiple times with different arguments. When
>>> invoked by run_kselftest.sh, they are invoked without arguments (and as
>>> a consequence, some fail immediately).
>>>
>>> Some test programs are only launched by run_vmtests.sh:
>>>
>>> test_vmalloc.sh
>>>
>>> And some test programs and only launched by run_kselftest.sh:
>>>
>>> khugepaged
>>> migration
>>> mkdirty
>>> transhuge-stress
>>> split_huge_page_test
>>> mdwe_test
>>> write_to_hugetlbfs
>>>
>>> Furthermore, run_vmtests.sh is invoked by run_kselftest.sh, so in this
>>> case all the test programs invoked by both scripts are run twice!
>>>
>>> Needless to say, this is a bit of a mess. In the absence of fully
>>> understanding the history here, it looks to me like the best solution is
>>> to launch ALL test programs from run_vmtests.sh, and ONLY invoke
>>> run_vmtests.sh from run_kselftest.sh. This way, we get full control over
>>> the parameters, each program is only invoked the intended number of
>>> times, and regardless of which script is used, the same tests get run in
>>> the same way.
>>>
>>> The only drawback is that if using run_kselftest.sh, it's top-level tap
>>> result reporting reports only a single test and it fails if any of the
>>> contained tests fail. I don't see this as a big deal though since we
>>> still see all the nested reporting from multiple layers. The other issue
>>> with this is that all of run_vmtests.sh must execute within a single
>>> kselftest timeout period, so let's increase that to something more
>>> suitable.
>>>
>>> In the Makefile, TEST_GEN_PROGS will compile and install the tests and
>>> will add them to the list of tests that run_kselftest.sh will run.
>>> TEST_GEN_FILES will compile and install the tests but will not add them
>>> to the test list. So let's move all the programs from TEST_GEN_PROGS to
>>> TEST_GEN_FILES so that they are built but not executed by
>>> run_kselftest.sh. Note that run_vmtests.sh is added to TEST_PROGS, which
>>> means it ends up in the test list. (the lack of "_GEN" means it won't be
>>> compiled, but simply copied).
>>>
>>> Signed-off-by: Ryan Roberts <ryan.roberts@....com>
>>> ---
>>
>> Acked-by: David Hildenbrand <david@...hat.com>
>
> Thanks for letting me know, David. Sorry for the late response, still
> catching up things.
>
> I used to justify that from mm/ itself that everything should be PROG, but
> I see that from higher level where TEST_GEN_FILE|PROG is really used this
> makes sense. As long as vm_utils.o will be properly linked I'll be happy
> enough..
Yep that's the case; I've set it up so that vm_utils.o is linked in for both
TEST_GEN_FILE and TEST_GEN_PROG binaries.
>
> Acked-by: Peter Xu <peterx@...hat.com>
Thanks!
>
> Thanks,
>
Powered by blists - more mailing lists