linux-kernel - Re: [KTAP V2 PATCH v2] ktap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6CCB5F6D-EC6C-452A-9D25-0D7B3F9739AB@kernel.org>
Date: Sun, 04 Feb 2024 14:03:51 +0100
From: Kees Cook <kees@...nel.org>
To: Rae Moar <rmoar@...gle.com>, frowand.list@...il.com, davidgow@...gle.com,
 keescook@...omium.org, Tim.Bird@...y.com, shuah@...nel.org,
 brendanhiggins@...gle.com, dlatypov@...gle.com
CC: tytso@...gle.com, gustavo.padovan@...labora.com,
 ricardo.canuelo@...labora.com, guillaume.tucker@...labora.com,
 corbet@....net, kernelci@...ts.linux.dev, linux-kselftest@...r.kernel.org,
 kunit-dev@...glegroups.com, linux-doc@...r.kernel.org,
 linux-kernel@...r.kernel.org
Subject: Re: [KTAP V2 PATCH v2] ktap_v2: add test metadata



On January 26, 2024 11:14:26 PM GMT+01:00, Rae Moar <rmoar@...gle.com> wrote:
> KTAP version 2
> # ktap_test: main
> # ktap_arch: uml
> 1..1
>     KTAP version 2
>     # ktap_test: suite_1
>     # ktap_subsystem: example
>     # ktap_test_file: lib/test.c

I think it's a mistake to mix "diagnostics" lines with semantic lines. Since the diagnostic prefix is [# ] (hash space) how about make the test metadata lines be [#:] (hash colon). For example:


     1..2
     ok 1 test_1
     #:ktap_test: test_2
     #:ktap_speed: very_slow
     #:custom_is_flaky: true
     # format-free stuff goes here
     ok 2 test_2
...

> ok 1 test_suite
>
>The changes to the KTAP specification outline the format, location, and
>different types of metadata.
>
>Here is a link to a version of the KUnit parser that is able to parse test
>metadata lines for KTAP version 2. Note this includes test metadata
>lines for the main level of KTAP.
>
>Link: https://kunit-review.googlesource.com/c/linux/+/5889
>
>Signed-off-by: Rae Moar <rmoar@...gle.com>
>---
> Documentation/dev-tools/ktap.rst | 163 ++++++++++++++++++++++++++++++-
> 1 file changed, 159 insertions(+), 4 deletions(-)
>
>diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst
>index ff77f4aaa6ef..4480eaf5bbc3 100644
>--- a/Documentation/dev-tools/ktap.rst
>+++ b/Documentation/dev-tools/ktap.rst
>@@ -17,19 +17,20 @@ KTAP test results describe a series of tests (which may be nested: i.e., test
> can have subtests), each of which can contain both diagnostic data -- e.g., log
> lines -- and a final result. The test structure and results are
> machine-readable, whereas the diagnostic data is unstructured and is there to

We even say it's unstructured... :)


>+prefix must not include spaces or the characters ":" or "_".

Why not _?

-- 
Kees Cook