linux-kernel - Re: [PATCH i-g-t v2 3/4] lib/igt_kmod: add compatibility for KUnit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGS_qxpCPTSRNAyejL0gS0pZAoVNp9SdFwPFkHRyn5yMuPGkLw@mail.gmail.com>
Date:   Mon, 19 Sep 2022 17:25:38 -0700
From:   Daniel Latypov <dlatypov@...gle.com>
To:     Isabella Basso <isabbasso@...eup.net>
Cc:     David Gow <davidgow@...gle.com>, igt-dev@...ts.freedesktop.org,
        Magali Lemes <magalilemes00@...il.com>,
        Maíra Canal <maira.canal@....br>,
        Tales Aparecida <tales.aparecida@...il.com>,
        Rodrigo Siqueira <rodrigo.siqueira@....com>,
        Melissa Wen <mwen@...lia.com>,
        André Almeida <andrealmeid@...eup.net>,
        Trevor Woerner <twoerner@...il.com>,
        leandro.ribeiro@...labora.com, n@...aprado.net,
        KUnit Development <kunit-dev@...glegroups.com>,
        Brendan Higgins <brendanhiggins@...gle.com>,
        Daniel Vetter <daniel@...ll.ch>,
        Shuah Khan <skhan@...uxfoundation.org>,
        linux-kselftest@...r.kernel.org,
        ML dri-devel <dri-devel@...ts.freedesktop.org>,
        daniel@...ishbar.org, kernel list <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH i-g-t v2 3/4] lib/igt_kmod: add compatibility for KUnit

On Mon, Sep 19, 2022 at 1:43 PM Isabella Basso <isabbasso@...eup.net> wrote:
> >> +        * "(K)TAP version XX" should be the first line on all (sub)tests as per
> >> +        * https://www.kernel.org/doc/html/latest/dev-tools/ktap.html#version-lines
> >> +        * but actually isn't, as it currently depends on whoever writes the
> >> +        * test to print this info
> >
> > FYI: we're really trying to fix cases of "missing version lines",
> > largely by making the kunit_test_suites() macro work in more
> > circumstances.
> >
> > So while it may be worth still handling the case where this is
> > missing, I don't think there are any tests in the latest versions of
> > the kernel which should have this missing.
>
> I’m not sure if I totally get how these work. Every time I run a KUnit test I
> get something like this: https://pastebin.com/7Ff31PMC
>
> As you can see it has been loaded as a module, just like we intend to do it
> from IGT, and I see no version lines whatsoever. Am I doing something wrong?

You're doing everything right.

The problem is we only print the version line for the *built-in* tests.
It never gets printed for tests in loadable modules.

Some more details below, if interested.
Specifically, it happens in
https://elixir.bootlin.com/linux/latest/C/ident/kunit_print_tap_header

What David is referring to is we had tests that weren't in modules,
but didn't use the normal built-in code path. Those were also missing
TAP version lines.
But KUnit needs to do better here in the case of modules.

The goal was that you'd be able to parse the result of module based
tests by something like
$ cat /sys/kernel/debug/kunit/*/results | ./tools/testing/kunit/kunit.py parse
but this doesn't work because of the lack of a version line.

If we add it to each module's test results, then we'll need to update
the parse to allow multiple verison lines, I think.

Daniel