linux-kernel - Re: Running kunit tests on platform devices

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <443632b6-c589-ef62-2385-3e8406680343@gmail.com>
Date:   Thu, 18 Aug 2022 15:08:01 -0300
From:   Tales Lelo da Aparecida <tales.aparecida@...il.com>
To:     Ramon Fried <rfried.dev@...il.com>, David Gow <davidgow@...gle.com>
Cc:     Brendan Higgins <brendanhiggins@...gle.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        KUnit Development <kunit-dev@...glegroups.com>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: Running kunit tests on platform devices

On 17/08/2022 09:19, Ramon Fried wrote:
> On Wed, Aug 17, 2022 at 7:43 AM David Gow <davidgow@...gle.com> wrote:
>>
>> On Mon, Aug 15, 2022 at 5:16 PM Ramon Fried <rfried.dev@...il.com> wrote:
>>>
>>> Hi.
>>> I implemented a test suite that checks a platform driver, that's the
>>> only way I can test interrupts behavior in the code.
>>> Because it's a platform, I can't use kunit_test_suite(), so I call
>>> __kunit_test_suites_init() as part of the platform driver probe
>>> function.
>>>
>>> This works fine but has the following problems.
>>> "TAP version 14" string is not printed and it's impossible to parse
>>> the results using the script.
>>> In addition, the suite is not displayed in /sys/kernel/debug/kunit.
>>>
>>> It would be my pleasure to provide a patch that fixes this, I just
>>> wanted to make sure that my testing strategy makes sense.
>>>
>>> Thanks,
>>> Ramon
>>>
>> Hi Ramon,
>>
>> Thanks for reaching out. Calling __kunit_test_suites_init() directly
>> is not something we'd recommend (and are trying desperately to remove
>> existing uses), so several of the issues re: the "TAP version 14"
>> string et al are side effects of calling what is supposed to be an
>> internal KUnit function manually.
>>
>> That being said, we definitely do want to make testing platform
>> drivers as convenient as possible. I'm not sure why kunit_test_suite()
>> doesn't work for you for platform drivers: are you just missing some
>> way of instantiating the device from within a test context?
>>
>> I know Brendan has experimented with some hardware faking code, for
>> which there's some example use here:
>> https://kunit-review.googlesource.com/c/linux/+/5275
>> (Note that you'll need to look at the other patches in the 'relation
>> chain' for dependencies.)
>>
>> Equally, I've experimented a bit with testing old soundcard drivers
>> (via a platform device) here, which may be an easier way to look
>> through:
>> https://github.com/sulix/linux/commit/4e1620c86553b9edde7f032318cf417dc13e4d26
>>
>> Note that neither of those examples are anything other than
>> experiments, so may not work as-is, or be ideal.
>>
>> Otherwise, we're always happy to accept patches, though again, if
>> there's any way of getting your tests working without a direct call to
>> __kunit_test_suites_init() --- even if that would require patches to
>> work --- that'd be preferable on our end.
>>
>> Cheers,
>> -- David
> Hi David,
> Thanks for replying.
> I looked at the examples you shared, and they all fake the actual device.
> My intention is to actually interact with the real device. - get a
> real interrupt, etc.
> Is it wrong, was the intention that the platform device be mocked ?
> Thanks,
> Ramon.
> 

Hi, Ramon,

I particularly don't condemn writing tests that require hardware, but 
they're best avoided, mostly to facilitate running those tests.

Can you share your code?
I would be happy to take a look if its not a problem for you!

Kind regards,
Tales