linux-kernel - Re: Linux Testing Microconference at LPC

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAFd5g46y36Rpm_oaRRmYYqL_0Jrj95rQVBn8EoFw3qpO9Or69g@mail.gmail.com>
Date:   Mon, 3 Jun 2019 01:59:44 -0700
From:   Brendan Higgins <brendanhiggins@...gle.com>
To:     Knut Omang <knut.omang@...cle.com>
Cc:     Sasha Levin <sashal@...nel.org>, shuah <shuah@...nel.org>,
        Dhaval Giani <dhaval.giani@...il.com>,
        Sasha Levin <alexander.levin@...rosoft.com>,
        Kevin Hilman <khilman@...libre.com>,
        Tim Bird <tbird20d@...il.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        "Carpenter,Dan" <dan.carpenter@...cle.com>,
        Matthew Wilcox <willy@...radead.org>,
        gustavo.padovan@...labora.co.uk, Dmitry Vyukov <dvyukov@...gle.com>
Subject: Re: Linux Testing Microconference at LPC

On Wed, May 22, 2019 at 9:55 PM Knut Omang <knut.omang@...cle.com> wrote:

Sorry for the delayed reply.

>
> On Wed, 2019-05-22 at 14:02 -0700, Brendan Higgins wrote:
> > On Wed, May 15, 2019 at 08:36:49PM -0400, Sasha Levin wrote:
> > > On Wed, May 15, 2019 at 04:44:19PM -0600, shuah wrote:
> > > > Hi Sasha and Dhaval,
> > > >
> > > > On 4/11/19 11:37 AM, Dhaval Giani wrote:
> > > > > Hi Folks,
> > > > >
> > > > > This is a call for participation for the Linux Testing microconference
> > > > > at LPC this year.
> > > > >
> > > > > For those who were at LPC last year, as the closing panel mentioned,
> > > > > testing is probably the next big push needed to improve quality. From
> > > > > getting more selftests in, to regression testing to ensure we don't
> > > > > break realtime as more of PREEMPT_RT comes in, to more stable distros,
> > > > > we need more testing around the kernel.
> > > > >
> > > > > We have talked about different efforts around testing, such as fuzzing
> > > > > (using syzkaller and trinity), automating fuzzing with syzbot, 0day
> > > > > testing, test frameworks such as ktests, smatch to find bugs in the
> > > > > past. We want to push this discussion further this year and are
> > > > > interested in hearing from you what you want to talk about, and where
> > > > > kernel testing needs to go next.
> > > > >
> > > > > Please let us know what topics you believe should be a part of the
> > > > > micro conference this year.
> > > > >
> > > > > Thanks!
> > > > > Sasha and Dhaval
> > > > >
> > > >
> > > > A talk on KUnit from Brendan Higgins will be good addition to this
> > > > Micro-conference. I am cc'ing Brendan on this thread.
> > > >
> > > > Please consider adding it.
> > >
> > > FWIW, the topic of unit tests is already on the schedule. There seems to
> > > be two different sub-topics here (kunit vs KTF) so there's a good
> > > discussion to be had here on many levels.
> >
> > Cool, so do we just want to go with that? Have a single slot for KUnit
> > and KTF combined?
> >
> > We can each present our work up to this point; maybe offer some
> > background and rationale on why we made the decision we have and then we
> > can have some moderated discussion on, pros, cons, next steps, etc?
>
> I definitely had KTF and KUnit in mind when proposing this topic.

Awesome!

> If you recall from the last time we discussed unit testing, each slot is
> fairly limited in time. My plan for the intro for discussion is to

Yeah, as per Steven's comment, I also submitted a refereed talk for
more detailed stuff.

> itemize some of the distinct goals we try to achieve with our frameworks and have a
> discussion based on that. In light of the discussion around your patch sets,

Sounds good to me. One thing I would like to talk about is maybe
trying to classify different categories of tests (unit vs. integration
vs. end-to-end), where they fit into the Linux kernel, how
prescriptivist should we be in categorization and what a test is for,
etc. I think this has been a point of disagreement/confusion on my
patchsets as well.

> one topic is also the question of whether a common API would be useful/desired,
> and whether we can "capture" a short namespace for that.

I am not opposed. This could potentially tie in to what kind of test
something is as I mentioned above. In anycase, sounds like there is a
lot of room for good discussion.

Thanks!