linux-kernel - Re: [PATCH] selftests/mm: Have the harness run each test category separately

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <56e2d214-fe48-48b0-88a2-287294520cb4@sirena.org.uk>
Date: Fri, 23 Jan 2026 12:35:06 +0000
From: Mark Brown <broonie@...nel.org>
To: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>,
	David Hildenbrand <david@...nel.org>,
	"Liam R. Howlett" <Liam.Howlett@...cle.com>,
	Vlastimil Babka <vbabka@...e.cz>, Mike Rapoport <rppt@...nel.org>,
	Suren Baghdasaryan <surenb@...gle.com>,
	Michal Hocko <mhocko@...e.com>, Shuah Khan <shuah@...nel.org>,
	Jason Gunthorpe <jgg@...pe.ca>, Leon Romanovsky <leon@...nel.org>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	linux-kselftest@...r.kernel.org
Subject: Re: [PATCH] selftests/mm: Have the harness run each test category
 separately

On Fri, Jan 23, 2026 at 12:09:52PM +0000, Lorenzo Stoakes wrote:
> On Fri, Jan 23, 2026 at 11:21:39AM +0000, Mark Brown wrote:
> > On Thu, Jan 22, 2026 at 07:13:06PM +0000, Lorenzo Stoakes wrote:

> > > # IMPORTANT: If you add a new test CATEGORY please add a simple wrapper
> > > # script so kunit knows to run it, and add it to the list below.
> > > # If you do not YOUR TESTS WILL NOT RUN IN THE CI.

> > Is the Makefile the place for that or run_vmtests.sh?  You don't need to
> > edit the Makefile to add a category.

> As I said, you have to edit the Make file to _add a new test_.

> The point being you will necessarily see this. It'd be fairly odd to add a new
> category without adding a new test.

You do, but adding the category is actually done in run_vmtests.sh and I
can see someone doing something like adding the test to the Makefile to
build the test while still developing it, then going back and hooking it 
up to run_vmtests.sh when they're satisfied that things work and
deciding at that point that it doesn't fit with any of the existing
categories.

> And since you'd need to update the Makefile, putting this comment at the top of
> the file immediately next to where you have to put it makes sense.

I guess.  At the top of the Makefile or right next to where you have to
add the tests?  There's about 50 lines of setup at the top of the
Makefile before you get to all the TEST_GEN_FILES.

> > > > +TEST_PROGS += ksft_vmalloc.sh

> > > Is this something that only kunit will interpret, or will it impact the
> > > build in any other way?

> > KUnit isn't involved here?  This is just how you specify which programs
> > are run by kselftest, this is a Makefile in the kselftest framework.

> s/kunit/kselftest/.

> I use the mm selftests locally by running sudo ./run_vmtests.sh directly. So I
> don't know how TEST_GEN_PROGS interacts with anything else.

If you're not invoking any of the kselftest infrastructure it's not
going to do anything, nothing execept make looks at the Makefile.  When
you do a build it'll result in the specified binary being built (and
installed if you ask for that) but not otherwise interacted with by the
generic framework.  See:

   https://docs.kernel.org/dev-tools/kselftest.html

for the various variables that the build system has for suite Makefiles.

Download attachment "signature.asc" of type "application/pgp-signature" (489 bytes)