linux-kernel - Re: [PATCH] kunit: tool: don't print out test statuses w/ 0s in summary

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGS_qxrpPquXVTOhDs4hvBfC-yBZKOCBGhEudh=PHz_tNCBwEA@mail.gmail.com>
Date:   Thu, 7 Apr 2022 22:54:11 -0500
From:   Daniel Latypov <dlatypov@...gle.com>
To:     David Gow <davidgow@...gle.com>
Cc:     Brendan Higgins <brendanhiggins@...gle.com>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        KUnit Development <kunit-dev@...glegroups.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH] kunit: tool: don't print out test statuses w/ 0s in summary

On Thu, Apr 7, 2022 at 10:48 PM David Gow <davidgow@...gle.com> wrote:
>
> On Fri, Apr 8, 2022 at 6:30 AM Daniel Latypov <dlatypov@...gle.com> wrote:
> >
> > Before:
> > > Testing complete. Passed: 137, Failed: 0, Crashed: 0, Skipped: 36, Errors: 0
> >
> > After:
> > > Testing complete. Passed: 137, Skipped: 36
> >
> > Even with our current set of statuses, the output is a bit verbose.
> > It could get worse in the future if we add more (e.g. timeout, kasan).
> > Let's only print the relevant ones.
> >
> > I had previously been sympathetic to the argument that always
> > printing out all the statuses would make it easier to parse results.
> > But now we have commit acd8e8407b8f ("kunit: Print test statistics on
> > failure"), there are test counts printed out in the raw output.
> > We don't currently print out an overall total across all suites, but it
> > would be easy to add, if we see a need for that.
> >
> > Signed-off-by: Daniel Latypov <dlatypov@...gle.com>
> > ---
>
> Looks good to me. I agree that we should add a total, too. I was
> thinking of adding one anyway, but now there's more space for it, I've
> just sent a patch out.

I was specifically referring to the test statistics in the kernel output.
We print out the counts per suite, but we don't print out the total count.

But a total in the kunit.py parsed output might be useful as well.

>
> Reviewed-by: David Gow <davidgow@...gle.com>
>
> Cheers,
> -- David