linux-kernel - Re: [PATCH v1 9/9] selftests/mm: Run all tests from run

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 13 Jul 2023 16:50:58 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Ryan Roberts <ryan.roberts@....com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Shuah Khan <shuah@...nel.org>,
        Jérôme Glisse <jglisse@...hat.com>,
        Mark Brown <broonie@...nel.org>,
        John Hubbard <jhubbard@...dia.com>,
        Florent Revest <revest@...omium.org>,
        "Liam R. Howlett" <Liam.Howlett@...cle.com>
Cc:     linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v1 9/9] selftests/mm: Run all tests from run_vmtests.sh

On 13.07.23 15:54, Ryan Roberts wrote:
> It is very unclear to me how one is supposed to run all the mm selftests
> consistently and get clear results.
> 
> Most of the test programs are launched by both run_vmtests.sh and
> run_kselftest.sh:
> 
>    hugepage-mmap
>    hugepage-shm
>    map_hugetlb
>    hugepage-mremap
>    hugepage-vmemmap
>    hugetlb-madvise
>    map_fixed_noreplace
>    gup_test
>    gup_longterm
>    uffd-unit-tests
>    uffd-stress
>    compaction_test
>    on-fault-limit
>    map_populate
>    mlock-random-test
>    mlock2-tests
>    mrelease_test
>    mremap_test
>    thuge-gen
>    virtual_address_range
>    va_high_addr_switch
>    mremap_dontunmap
>    hmm-tests
>    madv_populate
>    memfd_secret
>    ksm_tests
>    ksm_functional_tests
>    soft-dirty
>    cow
> 

Which run_kselftest.sh are you referring to, the one in the parent directory?

How to invoke it to run these mm tests?

(I never dared invoking something different than
run_vmtests.sh ;) )

[...]

> 
> Signed-off-by: Ryan Roberts <ryan.roberts@....com>
> ---
>   tools/testing/selftests/mm/Makefile       | 79 ++++++++++++-----------
>   tools/testing/selftests/mm/run_vmtests.sh | 23 +++++++
>   tools/testing/selftests/mm/settings       |  2 +-
>   3 files changed, 64 insertions(+), 40 deletions(-)
> 
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 66d7c07dc177..881ed96d96fd 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -35,39 +35,39 @@ MAKEFLAGS += --no-builtin-rules
>   CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
>   LDLIBS = -lrt -lpthread
>   
> -TEST_GEN_PROGS = cow
> -TEST_GEN_PROGS += compaction_test
> -TEST_GEN_PROGS += gup_longterm
> -TEST_GEN_PROGS += gup_test
> -TEST_GEN_PROGS += hmm-tests
> -TEST_GEN_PROGS += hugetlb-madvise
> -TEST_GEN_PROGS += hugepage-mmap
> -TEST_GEN_PROGS += hugepage-mremap
> -TEST_GEN_PROGS += hugepage-shm
> -TEST_GEN_PROGS += hugepage-vmemmap
> -TEST_GEN_PROGS += khugepaged
> -TEST_GEN_PROGS += madv_populate
> -TEST_GEN_PROGS += map_fixed_noreplace
> -TEST_GEN_PROGS += map_hugetlb
> -TEST_GEN_PROGS += map_populate
> -TEST_GEN_PROGS += memfd_secret
> -TEST_GEN_PROGS += migration
> -TEST_GEN_PROGS += mkdirty
> -TEST_GEN_PROGS += mlock-random-test
> -TEST_GEN_PROGS += mlock2-tests
> -TEST_GEN_PROGS += mrelease_test
> -TEST_GEN_PROGS += mremap_dontunmap
> -TEST_GEN_PROGS += mremap_test
> -TEST_GEN_PROGS += on-fault-limit
> -TEST_GEN_PROGS += thuge-gen
> -TEST_GEN_PROGS += transhuge-stress
> -TEST_GEN_PROGS += uffd-stress
> -TEST_GEN_PROGS += uffd-unit-tests
> -TEST_GEN_PROGS += soft-dirty
> -TEST_GEN_PROGS += split_huge_page_test
> -TEST_GEN_PROGS += ksm_tests
> -TEST_GEN_PROGS += ksm_functional_tests
> -TEST_GEN_PROGS += mdwe_test
> +TEST_GEN_FILES = cow
> +TEST_GEN_FILES += compaction_test
> +TEST_GEN_FILES += gup_longterm
> +TEST_GEN_FILES += gup_test
> +TEST_GEN_FILES += hmm-tests
> +TEST_GEN_FILES += hugetlb-madvise
> +TEST_GEN_FILES += hugepage-mmap
> +TEST_GEN_FILES += hugepage-mremap
> +TEST_GEN_FILES += hugepage-shm
> +TEST_GEN_FILES += hugepage-vmemmap
> +TEST_GEN_FILES += khugepaged
> +TEST_GEN_FILES += madv_populate
> +TEST_GEN_FILES += map_fixed_noreplace
> +TEST_GEN_FILES += map_hugetlb
> +TEST_GEN_FILES += map_populate
> +TEST_GEN_FILES += memfd_secret
> +TEST_GEN_FILES += migration
> +TEST_GEN_FILES += mkdirty
> +TEST_GEN_FILES += mlock-random-test
> +TEST_GEN_FILES += mlock2-tests
> +TEST_GEN_FILES += mrelease_test
> +TEST_GEN_FILES += mremap_dontunmap
> +TEST_GEN_FILES += mremap_test
> +TEST_GEN_FILES += on-fault-limit
> +TEST_GEN_FILES += thuge-gen
> +TEST_GEN_FILES += transhuge-stress
> +TEST_GEN_FILES += uffd-stress
> +TEST_GEN_FILES += uffd-unit-tests
> +TEST_GEN_FILES += soft-dirty
> +TEST_GEN_FILES += split_huge_page_test
> +TEST_GEN_FILES += ksm_tests
> +TEST_GEN_FILES += ksm_functional_tests
> +TEST_GEN_FILES += mdwe_test

IIRC, we recently converted all to TEST_GEN_PROGS. See

commit aef6fde75d8c6c1cad4a0e017a8d4cbee2143723
Author: Peter Xu <peterx@...hat.com>
Date:   Wed Apr 12 12:42:18 2023 -0400

     selftests/mm: use TEST_GEN_PROGS where proper
     
     TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to
     specify programs that need to build.  Logically all these binaries should
     all fall into TEST_GEN_PROGS.
     
     Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference
     all the tests easily later.


Why is that change required, and how does it interact with
run_kselftest.sh? (Not clear from you patch description.)

-- 
Cheers,

David / dhildenb