linux-kernel - Re: [PATCH] docs/kselftest: clarify running mainline tests on stables

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6f9083e2-d633-d483-702e-f974317133b7@intel.com>
Date:   Mon, 31 Jan 2022 13:20:18 -0800
From:   Reinette Chatre <reinette.chatre@...el.com>
To:     Shuah Khan <skhan@...uxfoundation.org>, <shuah@...nel.org>,
        <corbet@....net>
CC:     <linux-kselftest@...r.kernel.org>, <linux-doc@...r.kernel.org>,
        <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] docs/kselftest: clarify running mainline tests on stables

Hi Shuah,

On 1/31/2022 12:34 PM, Shuah Khan wrote:
> On 1/31/22 12:37 PM, Reinette Chatre wrote:
>> On 1/26/2022 12:13 PM, Shuah Khan wrote:
>>> Update the document to clarifiy support for running mainline
>>> kselftest on stable releases and the reasons for not removing
>>> test code that can test older kernels.
>>>
>>> Signed-off-by: Shuah Khan <skhan@...uxfoundation.org>
>>> ---
>>>   Documentation/dev-tools/kselftest.rst | 8 ++++++++
>>>   1 file changed, 8 insertions(+)
>>>
>>> diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
>>> index dcefee707ccd..a833ecf12fbc 100644
>>> --- a/Documentation/dev-tools/kselftest.rst
>>> +++ b/Documentation/dev-tools/kselftest.rst
>>> @@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code
>>>   paths in the kernel. Tests are intended to be run after building, installing
>>>   and booting a kernel.
>>>   +Kselftest from mainline can be run on older stable kernels. Running tests
>>> +from mainline offers the best coverage. Several test rings run mainline
>>> +kselftest suite on stable releases. The reason is that when a new test
>>> +gets added to test existing code to regression test a bug, we should be
>>> +able to run that test on an older kernel. Hence, it is important to keep
>>> +code that can still test an older kernel and make sure it skips the test
>>> +gracefully on newer releases.
>>> +
>>>   You can find additional information on Kselftest framework, how to
>>>   write new tests using the framework on Kselftest wiki:
>>>   
>>
>> (My apologies if this is already documented, I was not able to find this guidance
>> in Documentation/dev-tools/kselftest.rst nor when looking at the
>> "Kselftest use-cases..." slides linked from https://kselftest.wiki.kernel.org/)
>>
>> Could you please clarify what the requirement/expectation is regarding fixes
>> to tests? Since the recommendation in the above change is that Kselftest from
>> mainline should be run on older stable kernels, is it required to backport
>> fixes to the tests themselves to stable kernels?
>>
> 
> Couple of things to consider.
> 
> - A new test gets added to regression test a bug in stable and mainline
> - A new test gets added to test a kernel module/feature/API that has been
>   supported by stable and mainline releases
> 
> In both of these cases, running mainline kselftest on stables gives you the
> best coverage.
> 
> Kselftest fixes get pulled into stables like any other kernel fixes. If a few
> fixes are missing, it is a good idea to back-port if they fall into above two
> categories. If the test is for a new feature then, it doesn't make sense to
> back-port.
> 
> Hope this is helpful.

This is helpful, thank you very much. In summary I understand this to mean that
when testing a stable kernel it is recommended to run tests from mainline, but
running the tests from the same stable kernel version as the kernel being tested
is also a supported use case and thus fixes to tests should be back-ported. 

Thank you

Reinette