linux-kernel - Re: [PATCH v1 4/5] selftests/nolibc: report: extrude the test status line

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZKulgwwDjubkZuTa@1wt.eu>
Date:   Mon, 10 Jul 2023 08:30:27 +0200
From:   Willy Tarreau <w@....eu>
To:     Zhangjin Wu <falcon@...ylab.org>
Cc:     arnd@...db.de, linux-kernel@...r.kernel.org,
        linux-kselftest@...r.kernel.org, thomas@...ch.de
Subject: Re: [PATCH v1 4/5] selftests/nolibc: report: extrude the test status
 line

Hi Zhangjin,

On Mon, Jul 10, 2023 at 03:26:52AM +0800, Zhangjin Wu wrote:
> > On Thu, Jul 06, 2023 at 05:11:17PM +0800, Zhangjin Wu wrote:
> > > two newlines are added around the test summary line to extrude the test
> > > status.
> > 
> > But then we're back to making it annoying to check, having to figure
> > if we need to grep -A or grep -B etc. With grep 'status:' we would get
> > a synthetic status and the counters together. Why do you think it's
> > not convenient ? Or am I the only one considering it useful to just
> > run grep "status:" on all output files and figure a global status at
> > once ?
> 
> Sorry, Willy, my commit message may mislead you a little.
> 
> The newlines are added around the whole test summary line (with the
> status info), not only around the 'status info' ;-)

Ah OK, thanks for clarifying this!

> It is not for status grep, it is for developers to easily see the whole
> summary line at a glance

I understand but both work hand-in-hand, as every time you'll perform
a slight change, you'll necessarily rerun the whole series on all archs
to confirm, which is why I'm particularly annoying about the ability to
grep!

> And further, if not consider pure-text, the colors may be more helpful,
> for example, red for failed/failure, yellow for skipped/warning, green
> for passed/success, for example:
> 
>     $ echo | awk 'END{printf("138 test(s): \033[32m135\033[0m passed, \033[33m  2\033[0m skipped, \033[31m  1\033[0m failed => status: \033[31mfailure\033[0m\n");}'
>     138 test(s): 135 passed,   2 skipped,   1 failed => status: failure
> 
> But as we can see, the color control code is not readable and it may
> break the simple "status: failure" grep, we should use something like
> "status: .*failure" ;-)

Colors may only be used when stdout is a terminal, and still, some might
find it annonying (for example some distros use unreadably dark colors
that were apparently never tested over a black background, forcing users
to highlight the text by selecting it with the mouse to read it). Better
not start to play with this IMO, that's not really needed and may be more
annoying to some than helpful to most.

Thanks,
Willy