linux-kernel - Re: [PATCH] selftests: Install settings files to fix TIMEOUT failures

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87o8tsed6s.fsf@mpe.ellerman.id.au>
Date:   Fri, 21 Feb 2020 16:12:27 +1100
From:   Michael Ellerman <mpe@...erman.id.au>
To:     Shuah Khan <skhan@...uxfoundation.org>,
        Kees Cook <keescook@...omium.org>
Cc:     linux-kselftest@...r.kernel.org, linux-kernel@...r.kernel.org,
        Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH] selftests: Install settings files to fix TIMEOUT failures

Shuah Khan <skhan@...uxfoundation.org> writes:
> On 2/20/20 3:51 PM, Kees Cook wrote:
>> On Thu, Feb 20, 2020 at 03:42:41PM +1100, Michael Ellerman wrote:
>>> Commit 852c8cbf34d3 ("selftests/kselftest/runner.sh: Add 45 second
>>> timeout per test") added a 45 second timeout for tests, and also added
>>> a way for tests to customise the timeout via a settings file.
>>>
>>> For example the ftrace tests take multiple minutes to run, so they
>>> were given longer in commit b43e78f65b1d ("tracing/selftests: Turn off
>>> timeout setting").
>>>
>>> This works when the tests are run from the source tree. However if the
>>> tests are installed with "make -C tools/testing/selftests install",
>>> the settings files are not copied into the install directory. When the
>>> tests are then run from the install directory the longer timeouts are
>>> not applied and the tests timeout incorrectly.
>> 
>> Eek, yes, nice catch.
>> 
>>> So add the settings files to TEST_FILES of the appropriate Makefiles
>>> to cause the settings files to be installed using the existing install
>>> logic.
>> 
>> Instead, shouldn't lib.mk "notice" the settings file and automatically
>> include it in TEST_FILES instead of having to do this in each separate
>> Makefile?
>> 
>
> Let's keep it custom per test for now.

Yeah that seems less magical.

FWIW the patch below does work, so it's a small patch, but I'm not sure
it's worth the added complexity vs just listing the file in the few
tests that need it.

cheers


diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 1c8a1963d03f..82086c6ad5e7 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -20,6 +20,10 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS))
 TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED))
 TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES))
 
+ifeq ($(wildcard settings),settings)
+       TEST_FILES += settings
+endif
+
 ifdef KSFT_KHDR_INSTALL
 top_srcdir ?= ../../../..
 include $(top_srcdir)/scripts/subarch.include