linux-kernel - Re: [igt-dev] [PATCH i-g-t v2 0/4] Add support for KUnit tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 09 Sep 2022 16:49:24 +0200
From:   Janusz Krzysztofik <janusz.krzysztofik@...ux.intel.com>
To:     igt-dev@...ts.freedesktop.org
Cc:     linux-kselftest@...r.kernel.org, daniel@...ll.ch,
        kunit-dev@...glegroups.com, magalilemes00@...il.com,
        maira.canal@....br, dlatypov@...gle.com, n@...aprado.net,
        linux-kernel@...r.kernel.org, leandro.ribeiro@...labora.com,
        dri-devel@...ts.freedesktop.org, davidgow@...gle.com,
        tales.aparecida@...il.com, skhan@...uxfoundation.org,
        andrealmeid@...eup.net, Isabella Basso <isabbasso@...eup.net>,
        brendanhiggins@...gle.com, twoerner@...il.com,
        Isabella Basso <isabbasso@...eup.net>
Subject: Re: [igt-dev] [PATCH i-g-t v2 0/4] Add support for KUnit tests

Hi Isabella,

On Monday, 29 August 2022 02:09:16 CEST Isabella Basso wrote:
> This patch series was first developed as part of the LKCamp hackathon
> that happened last year[1], mainly focusing on refactoring DRM tests to
> use KUnit.
> 
> KUnit[2][3] is a unified test framework that provides helper tools,
> simplifying their development and execution. Using an x86-64 machine
> it's possible to run tests in the host's kernel natively using user-mode
> Linux[4] (aka UML), which simplifies usage in a wide variety of
> scenarios, including integration to CI.
> 
> As the tool's adoption widens into graphics testing territory, I and
> LKCamp members figured it would be important to support it in IGT, as
> it's a core tool for GPU drivers maintainers.
> 
> I have then added KUnit support into IGT mainly following the KTAP
> specs, and it can be tested using patch 4/4 in this series together with
> a DRM selftests patch series available at [5].

CI pre-merge results indicate that your new versions of tests were not able to 
load kunit module, then didn't actually execute any kunit tests, but returned 
SUCCESS.  That's probably not what we expect from IGT tests.

https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7698/shard-rkl-4/igt@drm_buddy.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7698/shard-rkl-1/igt@drm_mm.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7698/shard-rkl-5/igt@kms_selftest.html

Thanks,
Janusz

> 
> Changes since v1:
> - Major rework of parsing function structure:
>   - It is not longer recursive
>   - Adapt kselftests functions and structs to be used with KUnit
> - Switch DRM selftests to KUnit parsing as they're updated in the kernel
> - Replace AMD KUnit tests by DRM selftests
> 
> [1]: https://groups.google.com/g/kunit-dev/c/YqFR1q2uZvk/m/IbvItSfHBAAJ
> [2]: https://kunit.dev
> [3]: https://docs.kernel.org/dev-tools/kunit/index.html
> [4]: http://user-mode-linux.sourceforge.net
> [5]: https://lore.kernel.org/all/20220708203052.236290-1-maira.canal@usp.br/
> 
> Isabella Basso (4):
>   lib/igt_kmod: rename kselftest functions to ktest
>   lib/igt_kmod.c: check if module is builtin before attempting to unload
>     it
>   lib/igt_kmod: add compatibility for KUnit
>   tests: DRM selftests: switch to KUnit
> 
>  lib/igt_kmod.c       | 315 +++++++++++++++++++++++++++++++++++++++++--
>  lib/igt_kmod.h       |  14 +-
>  tests/drm_buddy.c    |   7 +-
>  tests/drm_mm.c       |   7 +-
>  tests/kms_selftest.c |  12 +-
>  5 files changed, 329 insertions(+), 26 deletions(-)
> 
>