lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <CABVgOSn3qVWUPAc8A6bKpt19OAsh5YXD=R1_OZKoSijB8v4+Rg@mail.gmail.com>
Date: Tue, 8 Feb 2022 15:17:14 +0800
From: David Gow <davidgow@...gle.com>
To: Frank Rowand <frowand.list@...il.com>
Cc: Jonathan Corbet <corbet@....net>,
Shuah Khan <skhan@...uxfoundation.org>,
Kees Cook <keescook@...omium.org>, Rae Moar <rmoar@...gle.com>,
"Bird, Tim" <Tim.Bird@...y.com>,
Brendan Higgins <brendanhiggins@...gle.com>,
Rae Moar <rmr167@...il.com>,
Guillaume Tucker <guillaume.tucker@...labora.com>,
Daniel Latypov <dlatypov@...gle.com>, kernelci@...ups.io,
KUnit Development <kunit-dev@...glegroups.com>,
"open list:KERNEL SELFTEST FRAMEWORK"
<linux-kselftest@...r.kernel.org>,
"open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v2 1/1] Documentation: dev-tools: clarify KTAP
specification wording
On Tue, Feb 8, 2022 at 12:28 AM <frowand.list@...il.com> wrote:
>
> From: Frank Rowand <frank.rowand@...y.com>
>
> Add the spec version to the title line.
>
> Explain likely source of "Unknown lines".
>
> "Unknown lines" in nested tests are optionally indented.
>
> Add "Unknown lines" items to differences between TAP & KTAP list
>
> Signed-off-by: Frank Rowand <frank.rowand@...y.com>
> ---
Thanks for fixing this. I'm happy with this version.
Reviewed-by: David Gow <davidgow@...gle.com>
>
> Changes since version 1
> - Explain likely source of "Unknown lines"
> - "Unknown line" in nested tests are optionally indented
> - Add "Unknown lines" items to differences between TAP & KTAP list
>
> Documentation/dev-tools/ktap.rst | 24 ++++++++++++++++++------
> 1 file changed, 18 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst
> index 878530cb9c27..9c40c94d3f12 100644
> --- a/Documentation/dev-tools/ktap.rst
> +++ b/Documentation/dev-tools/ktap.rst
> @@ -1,8 +1,8 @@
> .. SPDX-License-Identifier: GPL-2.0
>
> -========================================
> -The Kernel Test Anything Protocol (KTAP)
> -========================================
> +===================================================
> +The Kernel Test Anything Protocol (KTAP), version 1
> +===================================================
>
> TAP, or the Test Anything Protocol is a format for specifying test results used
> by a number of projects. It's website and specification are found at this `link
> @@ -174,6 +174,13 @@ There may be lines within KTAP output that do not follow the format of one of
> the four formats for lines described above. This is allowed, however, they will
> not influence the status of the tests.
>
> +This is an important difference from TAP. Kernel tests may print messages
> +to the system console or a log file. Both of these destinations may contain
> +messages either from unrelated kernel or userspace activity, or kernel
> +messages from non-test code that is invoked by the test. The kernel code
> +invoked by the test likely is not aware that a test is in progress and
> +thus can not print the message as a diagnostic message.
> +
> Nested tests
> ------------
>
> @@ -186,10 +193,13 @@ starting with another KTAP version line and test plan, and end with the overall
> result. If one of the subtests fail, for example, the parent test should also
> fail.
>
> -Additionally, all result lines in a subtest should be indented. One level of
> +Additionally, all lines in a subtest should be indented. One level of
> indentation is two spaces: " ". The indentation should begin at the version
> line and should end before the parent test's result line.
>
> +"Unknown lines" are not considered to be lines in a subtest and thus are
> +allowed to be either indented or not indented.
> +
> An example of a test with two nested subtests:
>
> .. code-block::
> @@ -225,9 +235,11 @@ Major differences between TAP and KTAP
> --------------------------------------
>
> Note the major differences between the TAP and KTAP specification:
> -- yaml and json are not recommended in diagnostic messages
> -- TODO directive not recognized
> +- yaml and json are not recommended in KTAP diagnostic messages
> +- TODO directive not recognized in KTAP
> - KTAP allows for an arbitrary number of tests to be nested
> +- TAP includes "Unknown lines" in the category of "Anything else"
> +- TAP says "Unknown lines" are "incorrect; KTAP allows "Unknown lines"
Nit: unmached quotes around "incorrect".
>
> The TAP14 specification does permit nested tests, but instead of using another
> nested version line, uses a line of the form
> --
> Frank Rowand <frank.rowand@...y.com>
>
Download attachment "smime.p7s" of type "application/pkcs7-signature" (4003 bytes)
Powered by blists - more mailing lists