linux-kernel - Re: [PATCH] kunit: tool: make parser preserve whitespace when printing test log

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAGS_qxr0AWCAT-wAw=fFkzryCc+MDZchi1kuRoRTSahzDQW_Ug@mail.gmail.com>
Date:   Wed, 7 Dec 2022 10:38:08 -0800
From:   Daniel Latypov <dlatypov@...gle.com>
To:     David Gow <davidgow@...gle.com>
Cc:     brendanhiggins@...gle.com, rmoar@...gle.com,
        linux-kernel@...r.kernel.org, kunit-dev@...glegroups.com,
        linux-kselftest@...r.kernel.org, skhan@...uxfoundation.org
Subject: Re: [PATCH] kunit: tool: make parser preserve whitespace when
 printing test log

On Tue, Dec 6, 2022 at 7:43 PM David Gow <davidgow@...gle.com> wrote:
>
> On Thu, Dec 1, 2022 at 2:54 AM 'Daniel Latypov' via KUnit Development

<snip>

> > After this change, here's what the output in context would look like
> > [13:40:46] =================== example (4 subtests) ===================
> > [13:40:46] # example_simple_test: initializing
> > [13:40:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29
> > [13:40:46] Expected 2 + 1 == 2, but
> > [13:40:46]     2 + 1 == 3 (0x3)
> > [13:40:46] [FAILED] example_simple_test
> > [13:40:46] [SKIPPED] example_skip_test
> > [13:40:46] [SKIPPED] example_mark_skipped_test
> > [13:40:46] [PASSED] example_all_expect_macros_test
> > [13:40:46]     # example: initializing suite
> > [13:40:46] # example: pass:1 fail:1 skip:2 total:4
> > [13:40:46] # Totals: pass:1 fail:1 skip:2 total:4
> > [13:40:46] ===================== [FAILED] example =====================
> >
> > This example shows one minor cosmetic defect this approach has.
> > The test counts lines prevent us from dedenting the suite-level output.
> > But at the same time, any form of non-KUnit output would do the same
> > unless it happened to be indented as well.
> >
> > Signed-off-by: Daniel Latypov <dlatypov@...gle.com>
> > ---
>
> I agree that this is a good idea.
>
> Personally, I think we could avoid some of the 'dedent' issues by
> disabling the test statistics lines when run via kunit.py (if
> raw_output is not enabled). That's probably better as a separate
> patch, though, so this looks good to go.

I was of a similar mindset initially, which is why I held off on
sending this patch out.
But as noted at the end of the commit desc, literally ~any kernel
output will cause this to happen.
And the whole point of this patch is to show such output, newly unmangled ;)

Secondly, only the kunit_{info,warn,err} logs are indented like this.
I think most test output would be coming from code outside the test
file, i.e. normal pr_info() calls.

But I figured I should still call it out somewhere in this email
though, since it does look a bit strange.
Maybe it didn't need to be as prominently as the commit desc.

Also, I was leaning towards just hiding it via some filtering in
kunit_parser.py, but Rae's reply [1] made me more unsure about which
way to go. So I'm thinking of leaving the lines in there for now.
[1] https://lore.kernel.org/linux-kselftest/CA+GJov6QKNy5fxkiu0P29WDtFgNs0GP4T27=X5Sva+V_BYc3+A@mail.gmail.com/

Daniel