Date:   Fri, 28 Feb 2020 11:43:33 -0700
From:   Shuah Khan <skhan@...uxfoundation.org>
To:     "Bird, Tim" <Tim.Bird@...y.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Cc:     Kevin Hilman <khilman@...libre.com>,
        Michael Ellerman <mpe@...erman.id.au>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Shuah Khan <skhan@...uxfoundation.org>
Subject: Re: [ANN] Kselftest integration into Kernel CI

On 2/28/20 10:50 AM, Bird, Tim wrote:
> 
> 
>> -----Original Message-----
>> From:  Shuah Khan
>>
>> Integrating Kselftest into Kernel CI rings depends on Kselftest build
>> and install framework to support Kernel CI use-cases. I am kicking off
>> an effort to support Kselftest runs in Kernel CI rings. Running these
>> tests in Kernel CI rings will help quality of kernel releases, both
>> stable and mainline.
>>
>> What is required for full support?
>>
>> 1. Cross-compilation & relocatable build support
>> 2. Generates objects in objdir/kselftest without cluttering main objdir
>> 3. Leave source directory clean
>> 4. Installs correctly in objdir/kselftest/kselftest_install and adds
>>      itself to run_kselftest.sh script generated during install.
>>
>> Note that install step is necessary for all files to be installed for
>> run time support.
>>
>> I looked into the current status and identified problems. The work is
>> minimal to add full support. Out of 80+ tests, 7 fail to cross-build
>> and 1 fails to install correctly.
>>
>> List is below:
>>
>> Tests fails to build: bpf, capabilities, kvm, memfd, mqueue, timens, vm
>> Tests fail to install: android (partial failure)
>> Leaves source directory dirty: bpf, seccomp
>>
>> I have patches ready for the following issues:
>>
>> Kselftest objects (test dirs) clutter top level object directory.
>> seccomp_bpf generates objects in the source directory.
>>
>> I created a topic branch to collect all the patches:
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/?h=kernelci
>>
>> I am going to start working on build problems. If anybody is
>> interested in helping me with this effort, don't hesitate to
>> contact me. I first priority is fixing build and install and
>> then look into tests that leave the source directory dirty.
> 
> I'm interested in this.  I'd like the same cleanups in order to run
> kselftest in Fuego, and I can try it with additional toolchains
> and boards.  Unfortunately, in terms of running tests, almost all
> the boards in my lab are running old kernels.  So the tests results
> aren't useful for upstream work.  But I can still test
> compilation and install issues, for the kselftest tests themselves.
> 

Testing compilation and install issues is very valuable. This is one
area that hasn't been test coverage compared to running tests. So it
great if you can help with build/install on linux-next to catch
problems in new tests. I am finding that older tests have been stable
and as new tests come in, we tend to miss catching these types of
problems.

Especially cross-builds and installs on arm64 and others.

>>
>> Detailed report can be found here:
>>
>> https://drive.google.com/file/d/11nnWOKIzzOrE4EiucZBn423lzSU_eNNv/view?usp=sharing
> 
> Is there anything you'd like me to look at specifically?  Do you want me to start
> at the bottom of the list and work up?  I could look at 'vm' or 'timens'.
> 

Yes you can start with vm and timens.

thanks,
-- Shuah

Powered by blists - more mailing lists