linux-kernel - Re: [PATCH v18 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ed26ac47-7f27-c3ec-56ae-8b3b9aa87654@kernel.org>
Date:   Mon, 7 Oct 2019 08:42:36 -0600
From:   shuah <shuah@...nel.org>
To:     Brendan Higgins <brendanhiggins@...gle.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>
Cc:     "Theodore Y. Ts'o" <tytso@....edu>,
        Frank Rowand <frowand.list@...il.com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Kees Cook <keescook@...gle.com>,
        Kieran Bingham <kieran.bingham@...asonboard.com>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Rob Herring <robh@...nel.org>, Stephen Boyd <sboyd@...nel.org>,
        Masahiro Yamada <yamada.masahiro@...ionext.com>,
        devicetree <devicetree@...r.kernel.org>,
        dri-devel <dri-devel@...ts.freedesktop.org>,
        kunit-dev@...glegroups.com,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        linux-nvdimm <linux-nvdimm@...ts.01.org>,
        linux-um@...ts.infradead.org,
        Sasha Levin <Alexander.Levin@...rosoft.com>,
        "Bird, Timothy" <Tim.Bird@...y.com>,
        Amir Goldstein <amir73il@...il.com>,
        Dan Carpenter <dan.carpenter@...cle.com>,
        Daniel Vetter <daniel@...ll.ch>, Jeff Dike <jdike@...toit.com>,
        Joel Stanley <joel@....id.au>,
        Julia Lawall <julia.lawall@...6.fr>,
        Kevin Hilman <khilman@...libre.com>,
        Knut Omang <knut.omang@...cle.com>,
        Logan Gunthorpe <logang@...tatee.com>,
        Michael Ellerman <mpe@...erman.id.au>,
        Petr Mladek <pmladek@...e.com>,
        Randy Dunlap <rdunlap@...radead.org>,
        Richard Weinberger <richard@....at>,
        David Rientjes <rientjes@...gle.com>,
        Steven Rostedt <rostedt@...dmis.org>, wfg@...ux.intel.com,
        shuah <shuah@...nel.org>
Subject: Re: [PATCH v18 00/19] kunit: introduce KUnit, the Linux kernel unit
 testing framework

On 10/7/19 2:40 AM, Brendan Higgins wrote:
> On Sun, Oct 6, 2019 at 10:18 AM Linus Torvalds
> <torvalds@...ux-foundation.org> wrote:
>>
>> On Sun, Oct 6, 2019 at 9:55 AM Theodore Y. Ts'o <tytso@....edu> wrote:
>>>
>>> Well, one thing we *can* do is if (a) if we can create a kselftest
>>> branch which we know is stable and won't change, and (b) we can get
>>> assurances that Linus *will* accept that branch during the next merge
>>> window, those subsystems which want to use kself test can simply pull
>>> it into their tree.
>>
>> Yes.
>>
>> At the same time, I don't think it needs to be even that fancy. Even
>> if it's not a stable branch that gets shared between different
>> developers, it would be good to just have people do a "let's try this"
>> throw-away branch to use the kunit functionality and verify that
>> "yeah, this is fairly convenient for ext4".
>>
>> It doesn't have to be merged in that form, but just confirmation that
>> the infrastructure is helpful before it gets merged would be good.
> 
> I thought we already had done this satisfactorily.
> 
Adding a couple more tests will only help in the long run. The idea is
to see can this help

> We have one proof-of-concept test in the branch in the kselftest repo
> (proc sysctl test) that went out in the pull request, and we also had
> some other tests that were not in the pull request (there is the ext4
> timestamp stuff mentioned above, and we also had one against the list
> data structure), which we were planning on sending out for review once
> Shuah's pull request was accepted. I know the apparmor people also
> wrote some tests that they said were useful; however, I have not
> coordinated with them on upstreaming their tests. I know of some other
> people who are using it, but I don't think the tests are as far along
> for upstreaming.
> 

Maybe that is a good start. To get the tests that are already in use
and get them in shape for upstream.

> The point is: I thought we had plenty of signal that KUnit would be
> useful to have merged into the mainline kernel. I thought the only
> reason it was rejected for 5.4 was due to the directory name issue
> combined with bad timing.
> 

That is probably the initial thought. However, it makes perfect sense
to add a couple of tests in. We have a few weeks anyway and it gives
us more confidence on kunit.

I already have a branch that is in linux-next and it just has kunit in
it and I will rebase it to 5.4-rc1.

https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/log/?h=test

Let's use that for kunit work for 5.5. I won't add any kselftest patches
to it and keep it dedicated for kunit work. When tests are ready for
upstream, I can keep adding them to this branch.

thanks,
-- Shuah