linux-kernel - Re: [PATCH 1/2] kunit: tool: make unit test not print parsed testdata to stdout

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGS_qxpghJUj+R74SFPjq=VDoNdVp7Wq2b+AcWwmHHeCQVjmPg@mail.gmail.com>
Date:   Mon, 31 Oct 2022 09:37:06 -0700
From:   Daniel Latypov <dlatypov@...gle.com>
To:     David Gow <davidgow@...gle.com>
Cc:     brendanhiggins@...gle.com, rmoar@...gle.com,
        linux-kernel@...r.kernel.org, kunit-dev@...glegroups.com,
        linux-kselftest@...r.kernel.org, skhan@...uxfoundation.org
Subject: Re: [PATCH 1/2] kunit: tool: make unit test not print parsed testdata
 to stdout

On Sat, Oct 29, 2022 at 8:31 PM David Gow <davidgow@...gle.com> wrote:
>
> On Sat, Oct 29, 2022 at 5:03 AM Daniel Latypov <dlatypov@...gle.com> wrote:
> >
> > Currently, if you run
> > $ ./tools/testing/kunit/kunit_tool_test.py
> > you'll see a lot of output from the parser as we feed it testdata.
> >
> > This makes the output hard to read and fairly confusing, esp. since our
> > testdata includes example failures, which get printed out in red.
> >
> > Silence that output so real failures are easier to see.
> >
> > Signed-off-by: Daniel Latypov <dlatypov@...gle.com>
> > ---
>
> Thanks -- this has been annoying me for ages.
>
> That being said, this isn't a perfect fix, the "usage" text and
> "Reconfiguring .config"  still show up for me:
> ---
> davidgow@...cestar:~/Development/linux-kselftest$
> ./tools/testing/kunit/kunit_tool_test.py
> ..............................usage: kunit_tool_test.py run [-h]
> [--build_dir DIR] [--make_options X=Y] [--alltests] [--kunitconfig
> PATHS] [--kconfig_add CONFIG_X=Y] [--arch ARCH] [--cross_compile
> PREFIX] [--qemu_config FILE] [--qemu_ar
> gs] [--jobs N]
>                              [--timeout SECONDS] [--kernel_args]
> [--run_isolated {suite,test}] [--raw_output [{all,kunit}]] [--json
> [FILE]]
>                              [filter_glob]
> kunit_tool_test.py run: error: argument --raw_output: invalid choice:
> 'invalid' (choose from 'all', 'kunit')
> ..............................Generating .config ...
> .Regenerating .config ...
> .........
> ----------------------------------------------------------------------
> Ran 70 tests in 0.232s
>
> OK
> ---
>
> That's still a significant improvement on what we had before, though, so:

Yeah, I was originally going to make this patch silence all the output.
But I figured I would focus this patch on just the code using our
kunit_printer abstraction, aka the parser.

We could address the other ones, but we'd have to mock out the
built-in print function.
As someone who is guilty of print(f) debugging, doing that means I no
longer have an easy of way of adding debugging code to those test
cases :P