linux-kernel - RFC: KTAP documentation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <d38bf9f9-8a39-87a6-8ce7-d37e4a641675@gmail.com>
Date:   Sun, 21 Jun 2020 17:45:44 -0500
From:   Frank Rowand <frowand.list@...il.com>
To:     "Bird, Tim" <Tim.Bird@...y.com>,
        "shuah@...nel.org" <shuah@...nel.org>,
        "linux-kselftest@...r.kernel.org" <linux-kselftest@...r.kernel.org>,
        Brendan Higgins <brendanhiggins@...gle.com>,
        David Gow <davidgow@...gle.com>,
        Kees Cook <keescook@...omium.org>,
        Paolo Bonzini <pbonzini@...hat.com>
Cc:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: RFC: KTAP documentation - expected messages

Tim Bird started a thread [1] proposing that he document the selftest result
format used by Linux kernel tests.  

[1] https://lore.kernel.org/r/CY4PR13MB1175B804E31E502221BC8163FD830@CY4PR13MB1175.namprd13.prod.outlook.com

The issue of messages generated by the kernel being tested (that are not
messages directly created by the tests, but are instead triggered as a
side effect of the test) came up.  In this thread, I will call these
messages "expected messages".  Instead of sidetracking that thread with
a proposal to handle expected messages, I am starting this new thread.

I implemented an API for expected messages that are triggered by tests
in the Devicetree unittest code, with the expectation that the specific
details may change when the Devicetree unittest code adapts the KUnit
API.  It seems appropriate to incorporate the concept of expected
messages in Tim's documentation instead of waiting to address the
subject when the Devicetree unittest code adapts the KUnit API, since
Tim's document may become the kernel selftest standard.

Instead of creating a very long email containing multiple objects,
I will reply to this email with a separate reply for each of:

  The "expected messages" API implemention and use can be from
  drivers/of/unittest.c in the mainline kernel.

  of_unittest_expect - A proof of concept perl program to filter console
                       output containing expected messages output

                       of_unittest_expect is also available by cloning
                       https://github.com/frowand/dt_tools.git

  An example raw console output with timestamps and expect messages.

  An example of console output processed by filter program
  of_unittest_expect to be more human readable.  The expected
  messages are not removed, but are flagged.

  An example of console output processed by filter program
  of_unittest_expect to be more human readable.  The expected
  messages are removed instead of being flagged.