[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <803bb28b-b25c-510d-bc70-b6726f750538@arm.com>
Date: Thu, 13 Jul 2023 16:46:32 +0100
From: Ryan Roberts <ryan.roberts@....com>
To: Mark Brown <broonie@...nel.org>
Cc: David Hildenbrand <david@...hat.com>,
Andrew Morton <akpm@...ux-foundation.org>,
Shuah Khan <shuah@...nel.org>,
Jérôme Glisse <jglisse@...hat.com>,
John Hubbard <jhubbard@...dia.com>,
Florent Revest <revest@...omium.org>,
"Liam R. Howlett" <Liam.Howlett@...cle.com>,
linux-kernel@...r.kernel.org, linux-mm@...ck.org,
linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v1 9/9] selftests/mm: Run all tests from run_vmtests.sh
On 13/07/2023 16:43, Mark Brown wrote:
> On Thu, Jul 13, 2023 at 04:36:18PM +0100, Ryan Roberts wrote:
>> On 13/07/2023 16:30, Mark Brown wrote:
>
>>> The results parsers I'm aware of like the LAVA one will DTRT with nested
>>> kselftests since that's required to pull see individual test cases run
>>> by a single binary so it's the common case to see at least one level of
>>> nesting.
>
>> That's good to hear. But bear in mind that run_vmtests.sh does not use TAP. So
>> you end up with a single top-level test who's result is reported with
>> run_kselftest.sh's TAP output. Then you have a second level (run_vmtests.sh)
>> using custom reporting, then _some_ of the tests invoked use TAP so you
>> sometimes have TAP at level 3. But those tests at level 2 that don't do their
>> own TAP output probably won't be parsed by LAVA?
>
> I think that should mostly mean that all the tests that don't
> individually produce KTAP output get ignored by parsers and those which
> do produce KTAP output will be seen as nesting one level up from where
> they are (ie, the individual cases will run directly from vmtest),
> though there's likely to be confusion about expected run numbers for
> things that actually pay attention to that.
I suspect it wouldn't be technically dififcult to add a --tap option to
run_vmtests.sh, which would switch the output format to TAP. If people are amenable.
>
>> Since you agreed to put this into the CI, I was going to call this part "your
>> problem" ;-)
>
> It'll run, the results are a different story. :P
Powered by blists - more mailing lists