linux-kernel - Re: [PATCH v18 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20191006165436.GA29585@mit.edu>
Date:   Sun, 6 Oct 2019 12:54:36 -0400
From:   "Theodore Y. Ts'o" <tytso@....edu>
To:     Brendan Higgins <brendanhiggins@...gle.com>
Cc:     shuah <shuah@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Frank Rowand <frowand.list@...il.com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Kees Cook <keescook@...gle.com>,
        Kieran Bingham <kieran.bingham@...asonboard.com>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Rob Herring <robh@...nel.org>, Stephen Boyd <sboyd@...nel.org>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        devicetree <devicetree@...r.kernel.org>,
        dri-devel <dri-devel@...ts.freedesktop.org>,
        kunit-dev@...glegroups.com,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        linux-nvdimm <linux-nvdimm@...ts.01.org>,
        linux-um@...ts.infradead.org,
        Sasha Levin <Alexander.Levin@...rosoft.com>,
        "Bird, Timothy" <Tim.Bird@...y.com>,
        Amir Goldstein <amir73il@...il.com>,
        Dan Carpenter <dan.carpenter@...cle.com>,
        Daniel Vetter <daniel@...ll.ch>, Jeff Dike <jdike@...toit.com>,
        Joel Stanley <joel@....id.au>,
        Julia Lawall <julia.lawall@...6.fr>,
        Kevin Hilman <khilman@...libre.com>,
        Knut Omang <knut.omang@...cle.com>,
        Logan Gunthorpe <logang@...tatee.com>,
        Michael Ellerman <mpe@...erman.id.au>,
        Petr Mladek <pmladek@...e.com>,
        Randy Dunlap <rdunlap@...radead.org>,
        Richard Weinberger <richard@....at>,
        David Rientjes <rientjes@...gle.com>,
        Steven Rostedt <rostedt@...dmis.org>, wfg@...ux.intel.com
Subject: Re: [PATCH v18 00/19] kunit: introduce KUnit, the Linux kernel unit
 testing framework

On Fri, Oct 04, 2019 at 06:18:04PM -0700, Brendan Higgins wrote:
> > Let's talk about current state. Right now kunit is in linux-next and
> > we want to add a few more tests. We will have to coordinate the effort.
> > Once kunit get into mainline, then the need for this coordination goes
> > down.
> 
> Sure, I was just thinking that getting other people to write the tests
> would be better. Since not only is it then useful to someone else, it
> provides the best possible exercise of KUnit.

Well, one thing we *can* do is if (a) if we can create a kselftest
branch which we know is stable and won't change, and (b) we can get
assurances that Linus *will* accept that branch during the next merge
window, those subsystems which want to use kself test can simply pull
it into their tree.

We've done this before in the file system world, when there has been
some common set of changes needed to improve, say, Direct I/O, where
the changes are put into a standalone branch, say, in the xfs tree,
and those file systems which need it as a building block can pull it
into their tree, and then add the changes needed to use those changes
into their file system git tree.  These changes are generally not
terribly controversial, and we've not had to worry about people want
to bikeshed the changes.

There is a risk with doing this of course, which is that if the branch
*is* controversial, or gets bike-shedded for some reason, then Linus
gets upset and any branches which depended on said branch will get
rejected at the next merge window.  Which is the requirement for (a)
and (b) above.  Presumably, the fact that people were unwilling to let
Kunit land during this merge window might will *because* we think more
changes might be pending?

The other thing I suppose I can do is to let the ext4 kunit tests land
in ext4 tree, but with the necessary #ifdef's around things like
"#include <kunit/test.h>" so that the build won't blow up w/o kunit
changes being in the tree that I'm building.  It means I won't be able
to run the tests without creating a test integration branch and
merging in kunit by hand, which will super-annoying, of course.  And
if some of the bike-shedding is in Kunit's interfaces, then that
becomes problematic as well, since any tests that are in ext4.git tree
might change if people want to rename Kunit's publically exported
functions (for example).

> Hey Ted, do you know if that ext4 timestamp test can go in through
> linux-kselftest? It seemed fairly self-contained. Or is that what you
> were saying wouldn't work for you?

Well, I was hoping that we might start creating more tests beyond just
the ext4 timestamp tests....

						- Ted