linux-kernel - Re: [PATCH] kunit: Adjust kunit_test timeout based on test

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CABVgOSm-ycyarsUZOkjt+O67GWJNv8qeTKOEDaBFsNA2_pqHRg@mail.gmail.com>
Date: Thu, 19 Jun 2025 12:45:41 +0800
From: David Gow <davidgow@...gle.com>
To: Rae Moar <rmoar@...gle.com>
Cc: Shuah Khan <skhan@...uxfoundation.org>, Ujwal Jain <ujwaljain@...gle.com>, 
	Geert Uytterhoeven <geert@...ux-m68k.org>, linux-kernel@...r.kernel.org, 
	linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com
Subject: Re: [PATCH] kunit: Adjust kunit_test timeout based on
 test_{suite,case} speed

On Thu, 19 Jun 2025 at 05:37, Rae Moar <rmoar@...gle.com> wrote:
>
> On Sat, Jun 14, 2025 at 4:47 AM David Gow <davidgow@...gle.com> wrote:
> >
> > From: Ujwal Jain <ujwaljain@...gle.com>
> >
> > Currently, the in-kernel kunit test case timeout is 300 seconds. (There
> > is a separate timeout mechanism for the whole test execution in
> > kunit.py, but that's unrelated.) However, tests marked 'slow' or 'very
> > slow' may timeout, particularly on slower machines.
> >
> > Implement a multiplier to the test-case timeout, so that slower tests
> > have longer to complete:
> > - DEFAULT -> 1x default timeout
> > - KUNIT_SPEED_SLOW -> 3x default timeout
> > - KUNIT_SPEED_VERY_SLOW -> 12x default timeout
>
> Hello!
>
> This change is looking great to me. No major concerns with the code
> and the tests are all passing.
>
> Just a few thoughts: I am wondering where the multipliers of 3 and 12
> came from? Are there specific tests that need those timeout amounts?
> And then given this changes the behavior of tests marked as slow and
> very_slow, we should update the documentation. And if possible, we
> should also add tests to check this feature.

The multipliers are pretty arbitrary, they're loosely based on the values in:
https://testing.googleblog.com/2010/12/test-sizes.html

There aren't any _specific_ tests which need these values, though, so
if there's another value which would be better, I'm all ears.
Regardless, the plan is to make the base timeout also configurable,
which should solve most machine/setup-specific, rather than
test-specific, issues.

For better documentation, that's a great idea. The in-kernel timeout
doesn't appear to be mentioned anywhere in the documentation.
So I'll see if we can add more complete documentation for this when we
make the default timeout configurable.

As for tests, I don't think we want to test the exact timeout values,
as that'd just make the test brittle. But testing that timeouts work
at all might work reliably. I'll have a play with them.

Cheers,
-- David

Download attachment "smime.p7s" of type "application/pkcs7-signature" (5281 bytes)