linux-kernel - Re: [KTAP V2 PATCH] ktap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 15 Mar 2023 16:45:29 -0500
From:   Frank Rowand <frowand.list@...il.com>
To:     Mark Brown <broonie@...nel.org>, kernelci@...ups.io,
        rmoar@...gle.com
Cc:     "Bird, Tim" <Tim.Bird@...y.com>,
        "davidgow@...gle.com" <davidgow@...gle.com>,
        "skhan@...uxfoundation.org" <skhan@...uxfoundation.org>,
        "keescook@...omium.org" <keescook@...omium.org>,
        "brendanhiggins@...gle.com" <brendanhiggins@...gle.com>,
        "corbet@....net" <corbet@....net>,
        "guillaume.tucker@...labora.com" <guillaume.tucker@...labora.com>,
        "dlatypov@...gle.com" <dlatypov@...gle.com>,
        "kunit-dev@...glegroups.com" <kunit-dev@...glegroups.com>,
        "linux-kselftest@...r.kernel.org" <linux-kselftest@...r.kernel.org>,
        "linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [KTAP V2 PATCH] ktap_v2: add skip test result

On 3/15/23 07:53, Mark Brown wrote:
> On Tue, Mar 14, 2023 at 06:03:59PM -0400, Rae Moar via groups.io wrote:
> 
>> One thing to note on the created churn: I have noticed a proportion of
>> kselftests currently implement skipped tests in a way that does not
>> use the SKIP directive. They use a comment of the format "# [SKIP]"
>> prior to a test result line with no SKIP directive. Thus, in order to
>> reach KTAP compliance the way skip tests are handled would need to be
>> changed in these cases anyways.
> 
> This is the documented way of reporting a skip in KTAP:
> 
>    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/dev-tools/ktap.rst#n97
> 
> TBH I'm finding it really hard to summon much enthusiasm for changing
> this except as part of some other incompatible update - the current
> format isn't ideal but deploying a change would be a bunch of hassle for
> the existing test automation systems.

Yes, there is no need to do a single specification change that results
in incompatibility.  But given the previous discussions there seem to
be plenty of other desired changes that will result in incompatibility.

My desire is to take our time to capture as much of the desired changes
as possible in version 2 of the specification.  And an expectation that
there will not need to be a version 3 of the specification for many years.
I will support not rushing version 2 of the specification so that this
goal can be realized.