linux-kernel - Re: [PATCH v2] rcu/torture: Improve badness extraction from console logs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAEXW_YTjDM9LWKiupgOXwfsz7RnNT24UEQ=RZB+3n1VdJRNqvw@mail.gmail.com>
Date:   Fri, 8 Sep 2023 09:53:30 -0400
From:   Joel Fernandes <joel@...lfernandes.org>
To:     paulmck@...nel.org
Cc:     linux-kernel@...r.kernel.org,
        Josh Triplett <josh@...htriplett.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        Lai Jiangshan <jiangshanlai@...il.com>,
        Shuah Khan <shuah@...nel.org>, rcu@...r.kernel.org,
        linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v2] rcu/torture: Improve badness extraction from console logs

On Fri, Sep 8, 2023 at 8:08 AM Paul E. McKenney <paulmck@...nel.org> wrote:
>
> On Fri, Sep 08, 2023 at 03:30:46AM +0000, Joel Fernandes (Google) wrote:
> > Currently console.log.diags contains an output like follows:
> > [ 2457.293734] WARNING: CPU: 2 PID: 13 at kernel/rcu/tasks.h:1061 rcu_tasks_trace_pregp_step+0x4a/0x50
> > [ 2457.542385] Call Trace:
> >
> > This is not very useful and easier access to the call trace is desired.
> > Improve the script by extracting more lines after each grep match.
> > Provide a summary in the beginning like before, but also include details
> > below. Limit the total number of issues to a maximum of 10. And limit
> > the lines included after each issue to a maximum of 20.
>
> Much better!
>
> The above commit log would be better something as follows:

Looks good to me, I will update it to use your version, thanks.

> ------------------------------------------------------------------------
>
> Plus please see one simplification below.
>
>                                                         Thanx, Paul
>
> > With these changes the output becomes:
> >
> > Issues found:
> > Line 6228: [ 2457.293734] WARNING: CPU: 2 PID: 13 at kernel/rcu/tasks.h:1061 rcu_tasks_trace_pregp_step+0x4a/0x50
> > Line 6245: [ 2457.542385] Call Trace:
> >
[...]
> >
> > Signed-off-by: Joel Fernandes (Google) <joel@...lfernandes.org>
> > ---
> > v1->v2: Limit number of issues reported and include summary on the top.
> >
> >  .../rcutorture/bin/console-badness.sh         | 42 ++++++++++++++++++-
> >  1 file changed, 41 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/testing/selftests/rcutorture/bin/console-badness.sh b/tools/testing/selftests/rcutorture/bin/console-badness.sh
> > index aad51e7c0183..2612a4931723 100755
> > --- a/tools/testing/selftests/rcutorture/bin/console-badness.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/console-badness.sh
> > @@ -9,10 +9,50 @@
> >  # Copyright (C) 2020 Facebook, Inc.
> >  #
> >  # Authors: Paul E. McKenney <paulmck@...nel.org>
> > +INPUT_DATA=$(< /dev/stdin)
>
> Long runs can have extremely large console.log files, which might not
> play well with shell variables.  So this really needs to go into a file.

Thanks a lot for this feedback. Just for completeness -- it is the
grepped lines (containing bad strings) that go into a shell variable,
not the whole file. That can still be quite large so your suggestion
is well taken and I will make the changes as you suggested!

[...]
> One issue here is that if a number of badness strings occur in console.log
> in quick succession, there will be overlapping output, as in a given line
> from console.log might be printed many times.  For example, "WARNING"
> is often almost immediately followed by "Call Trace".
>
> In that case, is there a nice way to print the lines starting with the
> line containing "WARNING" though the 20 lines following "Call Trace"?

Yes, maybe I can coalesce them somehow since we do have the line
numbers. Thanks for the suggestion! I did notice that too but now that
you mention, I will improve on that.

Will post another version soon, thanks!

 - Joel