lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 13 Jul 2023 16:36:18 +0100
From:   Ryan Roberts <ryan.roberts@....com>
To:     Mark Brown <broonie@...nel.org>
Cc:     David Hildenbrand <david@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Shuah Khan <shuah@...nel.org>,
        Jérôme Glisse <jglisse@...hat.com>,
        John Hubbard <jhubbard@...dia.com>,
        Florent Revest <revest@...omium.org>,
        "Liam R. Howlett" <Liam.Howlett@...cle.com>,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v1 9/9] selftests/mm: Run all tests from run_vmtests.sh

On 13/07/2023 16:30, Mark Brown wrote:
> On Thu, Jul 13, 2023 at 04:04:44PM +0100, Ryan Roberts wrote:
> 
>> So with this change at the kselftest level, there is a single test in its list;
>> run_vmtests.sh. And all the other tests that were previously in that list are
>> moved into run_vmtests.sh (if they weren't there already).
> 
> The results parsers I'm aware of like the LAVA one will DTRT with nested
> kselftests since that's required to pull see individual test cases run
> by a single binary so it's the common case to see at least one level of
> nesting.

That's good to hear. But bear in mind that run_vmtests.sh does not use TAP. So
you end up with a single top-level test who's result is reported with
run_kselftest.sh's TAP output. Then you have a second level (run_vmtests.sh)
using custom reporting, then _some_ of the tests invoked use TAP so you
sometimes have TAP at level 3. But those tests at level 2 that don't do their
own TAP output probably won't be parsed by LAVA?

Since you agreed to put this into the CI, I was going to call this part "your
problem" ;-)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ