linux-kernel - Re: [PATCH] kunit: Fix result propagation for parameterised tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <8a7e6806-97f5-6f13-5e97-328cf0d3a95d@linuxfoundation.org>
Date:   Fri, 11 Jun 2021 11:44:01 -0600
From:   Shuah Khan <skhan@...uxfoundation.org>
To:     Marco Elver <elver@...gle.com>, David Gow <davidgow@...gle.com>,
        Brendan Higgins <brendanhiggins@...gle.com>
Cc:     Arpitha Raghunandan <98.arpi@...il.com>,
        KUnit Development <kunit-dev@...glegroups.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [PATCH] kunit: Fix result propagation for parameterised tests

On 6/11/21 2:29 AM, Marco Elver wrote:
> On Fri, 11 Jun 2021 at 05:57, David Gow <davidgow@...gle.com> wrote:
>>
>> When one parameter of a parameterised test failed, its failure would be
>> propagated to the overall test, but not to the suite result (unless it
>> was the last parameter).
>>
>> This is because test_case->success was being reset to the test->success
>> result after each parameter was used, so a failing test's result would
>> be overwritten by a non-failing result. The overall test result was
>> handled in a third variable, test_result, but this was disacarded after
>> the status line was printed.
>>
>> Instead, just propagate the result after each parameter run.
>>
>> Signed-off-by: David Gow <davidgow@...gle.com>
>> Fixes: fadb08e7c750 ("kunit: Support for Parameterized Testing")
> 
> Reviewed-by: Marco Elver <elver@...gle.com>
> 
> Would Cc: stable be appropriate?
> 
> Thanks,
> -- Marco
> 
>> ---
>>
>> This is fixing quite a serious bug where some test suites would appear
>> to succeed even if some of their component tests failed. It'd be nice to
>> get this into kunit-fixes ASAP.
>>

Will apply this with cc stable.

>> (This will require a rework of some of the skip tests work, for which
>> I'll send out a new version soon.)
>>

Thanks for the heads up. I will wait for new version.

thanks,
-- Shuah