linux-kernel - Re: [PATCH 2/3] bitmap: Rename module

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <75a2960e-d489-4440-a8e1-487a7f84902e@linuxfoundation.org>
Date: Tue, 30 Jul 2024 12:17:12 -0600
From: Shuah Khan <skhan@...uxfoundation.org>
To: David Gow <davidgow@...gle.com>, Randy Dunlap <rdunlap@...radead.org>,
 kees@...nel.org, Muhammad Usama Anjum <usama.anjum@...labora.com>
Cc: Yury Norov <yury.norov@...il.com>,
 Andrew Morton <akpm@...ux-foundation.org>,
 Rasmus Villemoes <linux@...musvillemoes.dk>, Shuah Khan <shuah@...nel.org>,
 linux-kernel@...r.kernel.org, linux-kselftest@...r.kernel.org,
 John Hubbard <jhubbard@...dia.com>, kernel@...labora.com,
 Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH 2/3] bitmap: Rename module

On 7/30/24 09:55, Shuah Khan wrote:
> On 7/30/24 04:10, David Gow wrote:
>> On Mon, 29 Jul 2024 at 22:09, Randy Dunlap <rdunlap@...radead.org> wrote:
>>>
>>>
>>>
>>> On 7/29/24 1:07 AM, Muhammad Usama Anjum wrote:
>>>> On 7/27/24 10:35 PM, Yury Norov wrote:
>>>>> On Fri, Jul 26, 2024 at 04:06:57PM +0500, Muhammad Usama Anjum wrote:
>>>>>> Rename module to bitmap_kunit and rename the configuration option
>>>>>> compliant with kunit framework.
>>>>>
>>>>> ... , so those enabling bitmaps testing in their configs by setting
>>>>> "CONFIG_TEST_BITMAP=y" will suddenly get it broken, and will likely
>>>>> not realize it until something nasty will happen.
>>>> CONFIG_TEST_BITMAP was being enabled by the kselftest suite lib. The bitmap
>>>> test and its config option would disappear. The same test can be run by
>>>> just enabling KUNIT default config option:
>>>>
>>>> KUNIT_ALL_TESTS=y enables this bitmap config by default.
>>>>
>>>>>
>>>>> Sorry, NAK for config rename.
>>>>>
>>>
>>> I agree with Yury. Using KUNIT takes away test coverage for people who
>>> are willing to run selftests but not use KUNIT.
> 
> This is incorrect. There are selftest that are used to
> 
> - regression test a subsystem or a abi during boot
> - spot check on a running system to debug and test by loading
>    a test module.
> 
>>
>> I can see the point that renaming the config option is just churn, but
>> is there a reason people would run the bitmap selftest but be unable
>> or unwilling to use KUnit?
>>
>> Beyond a brief period of adjustment (which could probably be made
>> quite minimal with a wrapper script or something), there shouldn't
>> really be any fundamental difference: KUnit tests can already run at
>> boot, be configured with a config option, and write output to the
>> kernel log. There's nothing really being taken away here, and the
>> bonus of having easier access to run the tests with KUnit's tooling
>> (or have them automatically run by systems which run KUnit tests)
>> would seem worthwhile to me, especially since it's optional. And
>> CONFIG_KUNIT shouldn't be heavy enough to cause problems.
>>

Shouldn't be is the operative word? This doesn't help people who
want run a run bitmap test on a running system. This is a wrong
direction to go to say all testing has to be done under kunit.

What happened to the effort to run selftests as is under KUnit? What
is the motivation to convert all tests to kunit instead of trying to
provide support to run kselftest under kunit environment?

We discussed this a few years ago as I recall. Let's work on that
instead of removing existing selftests and regressing current use-cases?

Can we look into providing:

1. running kselftest under kunit environment without changes
    as user space applications?
2. Leave kselftests alone so we don't weaken kernel testing


thanks,
-- Shuah