[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <5a12536a-2e33-7a68-6cfd-fd991ddf875a@redhat.com>
Date: Mon, 17 Jul 2023 19:27:13 +0200
From: David Hildenbrand <david@...hat.com>
To: Ryan Roberts <ryan.roberts@....com>,
Andrew Morton <akpm@...ux-foundation.org>,
Shuah Khan <shuah@...nel.org>,
Jérôme Glisse <jglisse@...hat.com>,
Mark Brown <broonie@...nel.org>,
John Hubbard <jhubbard@...dia.com>,
Florent Revest <revest@...omium.org>,
Peter Xu <peterx@...hat.com>
Cc: linux-kernel@...r.kernel.org, linux-mm@...ck.org,
linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v2 8/8] selftests/mm: Run all tests from run_vmtests.sh
On 17.07.23 12:31, Ryan Roberts wrote:
> It is very unclear to me how one is supposed to run all the mm selftests
> consistently and get clear results.
>
> Most of the test programs are launched by both run_vmtests.sh and
> run_kselftest.sh:
>
> hugepage-mmap
> hugepage-shm
> map_hugetlb
> hugepage-mremap
> hugepage-vmemmap
> hugetlb-madvise
> map_fixed_noreplace
> gup_test
> gup_longterm
> uffd-unit-tests
> uffd-stress
> compaction_test
> on-fault-limit
> map_populate
> mlock-random-test
> mlock2-tests
> mrelease_test
> mremap_test
> thuge-gen
> virtual_address_range
> va_high_addr_switch
> mremap_dontunmap
> hmm-tests
> madv_populate
> memfd_secret
> ksm_tests
> ksm_functional_tests
> soft-dirty
> cow
>
> However, of this set, when launched by run_vmtests.sh, some of the
> programs are invoked multiple times with different arguments. When
> invoked by run_kselftest.sh, they are invoked without arguments (and as
> a consequence, some fail immediately).
>
> Some test programs are only launched by run_vmtests.sh:
>
> test_vmalloc.sh
>
> And some test programs and only launched by run_kselftest.sh:
>
> khugepaged
> migration
> mkdirty
> transhuge-stress
> split_huge_page_test
> mdwe_test
> write_to_hugetlbfs
>
> Furthermore, run_vmtests.sh is invoked by run_kselftest.sh, so in this
> case all the test programs invoked by both scripts are run twice!
>
> Needless to say, this is a bit of a mess. In the absence of fully
> understanding the history here, it looks to me like the best solution is
> to launch ALL test programs from run_vmtests.sh, and ONLY invoke
> run_vmtests.sh from run_kselftest.sh. This way, we get full control over
> the parameters, each program is only invoked the intended number of
> times, and regardless of which script is used, the same tests get run in
> the same way.
>
> The only drawback is that if using run_kselftest.sh, it's top-level tap
> result reporting reports only a single test and it fails if any of the
> contained tests fail. I don't see this as a big deal though since we
> still see all the nested reporting from multiple layers. The other issue
> with this is that all of run_vmtests.sh must execute within a single
> kselftest timeout period, so let's increase that to something more
> suitable.
>
> In the Makefile, TEST_GEN_PROGS will compile and install the tests and
> will add them to the list of tests that run_kselftest.sh will run.
> TEST_GEN_FILES will compile and install the tests but will not add them
> to the test list. So let's move all the programs from TEST_GEN_PROGS to
> TEST_GEN_FILES so that they are built but not executed by
> run_kselftest.sh. Note that run_vmtests.sh is added to TEST_PROGS, which
> means it ends up in the test list. (the lack of "_GEN" means it won't be
> compiled, but simply copied).
>
> Signed-off-by: Ryan Roberts <ryan.roberts@....com>
> ---
Acked-by: David Hildenbrand <david@...hat.com>
--
Cheers,
David / dhildenb
Powered by blists - more mailing lists