linux-kernel - Re: Linux Testing Microconference at LPC

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <f08575cfa7aae2c0d156ebbde75d7f6d8baefedf.camel@oracle.com>
Date:   Tue, 23 Apr 2019 10:37:51 +0200
From:   Knut Omang <knut.omang@...cle.com>
To:     Dhaval Giani <dhaval.giani@...il.com>,
        Sasha Levin <alexander.levin@...rosoft.com>,
        shuah <shuah@...nel.org>, Kevin Hilman <khilman@...libre.com>,
        Tim Bird <tbird20d@...il.com>,
        LKML <linux-kernel@...r.kernel.org>
Cc:     Steven Rostedt <rostedt@...dmis.org>,
        "Carpenter,Dan" <dan.carpenter@...cle.com>, willy@...radead.org,
        gustavo.padovan@...labora.co.uk, Dmitry Vyukov <dvyukov@...gle.com>
Subject: Re: Linux Testing Microconference at LPC

Hi,

On Thu, 2019-04-11 at 10:37 -0700, Dhaval Giani wrote:
> Hi Folks,
> 
> This is a call for participation for the Linux Testing microconference
> at LPC this year.
> 
> For those who were at LPC last year, as the closing panel mentioned,
> testing is probably the next big push needed to improve quality. From
> getting more selftests in, to regression testing to ensure we don't
> break realtime as more of PREEMPT_RT comes in, to more stable distros,
> we need more testing around the kernel.
> 
> We have talked about different efforts around testing, such as fuzzing
> (using syzkaller and trinity), automating fuzzing with syzbot, 0day
> testing, test frameworks such as ktests, smatch to find bugs in the
> past. We want to push this discussion further this year and are
> interested in hearing from you what you want to talk about, and where
> kernel testing needs to go next.
> 
> Please let us know what topics you believe should be a part of the
> micro conference this year.

I'd like to propose another topic on unit test framework 
support in the kernel:

>From the initial reactions and interest I have seen wrt. KTF 
(http://heim.ifi.uio.no/~knuto/ktf/, https://github.com/oracle/ktf) 
and the discussions on LKML around KUnit (https://lkml.org/lkml/2018/11/29/82), 
it seems there's a general belief that some form of unit test framework 
like these can be a good addition to the tools and infrastructure already available
in the kernel.

It seems however that different people have different notions about what 
and how such a framework should ideally look, and what features belong there.
I'd like to see if we can bring that discussion forward by focusing on 
some of these items, where people seem to have quite differing views
depending on where they come from. Here is a non extensive list of 
some topics that seems to pop up when this gets discussed:

  - "Purity" of unit testing - what constitutes a "unit" in the kernel?
  - Testing kernel code - user space vs kernel space? (both useful)
  - Immediate development/debugging requirements vs longer term needs
  - Driver/hardware interaction testing?
  - "Neat"-factor
  - ease of use
  - Network testing (more than 1 kernel involved)
  ...

I'd like to make a short intro into this, and hopefully we can have some 
good exchange based on that. 

[While our contribution (KTF) is currently available on Github, 
at that point in time I plan for us to have submitted a version 
of it to the LKML as well]

Thanks,
Knut