linux-kernel - Re: [Announce] LPC 2018: Testing and Fuzzing Microconference

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CACT4Y+ZN0OcTkY0ahvF0CGZv1bbLHGH2aEbCFuQzpC498cNojw@mail.gmail.com>
Date:   Mon, 24 Sep 2018 15:42:27 +0200
From:   Dmitry Vyukov <dvyukov@...gle.com>
To:     Matthew Wilcox <willy@...radead.org>
Cc:     Dhaval Giani <dhaval.giani@...il.com>,
        Sasha Levin <alexander.levin@...rosoft.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        alice.ferrazzi@...il.com, Kevin Hilman <khilman@...libre.com>,
        Tim Bird <tbird20d@...il.com>,
        Laura Abbott <labbott@...hat.com>,
        Steven Rostedt <rostedt@...dmis.org>,
        gustavo.padovan@...labora.co.uk,
        "Carpenter,Dan" <dan.carpenter@...cle.com>, knut.omang@...cle.com
Subject: Re: [Announce] LPC 2018: Testing and Fuzzing Microconference

On Sat, Sep 22, 2018 at 2:52 PM, Matthew Wilcox <willy@...radead.org> wrote:
> On Wed, Sep 19, 2018 at 10:13:15AM -0700, Dhaval Giani wrote:
>> Sasha and I are pleased to announce the Testing and Fuzzing track at
>> LPC [ 1 ]. We are planning to continue the discussions from last
>> year's microconference [2]. Many discussions from the Automated
>> Testing Summit [3] will also continue, and a final agenda will come up
>> only soon after that.
>>
>> Suggested Topics
>>
>> - Syzbot/syzkaller
>> - ATS
>> - Distro/stable testing
>> - kernelci
>> - kernelci auto bisection
>> - Unit testing framework
>>
>> We look forward to other interesting topics for this microconference
>> as a reply to this email.
>
> I would like to talk about the IDA test suite that was recently merged.
> See lib/test_ida.c.  It can be built as a module (CONFIG_TEST_IDA=m),
> built-in (=y) or built in userspace (as part of the radix tree test
> suite for historical reasons) along with the current kernel code.
>
> Being able to build the test suite in userspace allows for much more
> rapid development.  Building it in kernel space offers testing across
> a wide range of configurations that I don't have access to and can't
> necessarily simulate well in userspace.
>
> My userspace implementation of kmalloc() simulates failures (in a rather
> heavy-handed way; every non-GFP_KERNEL allocation fails).  That's rather
> harder to simulate in kernel space.  I'd like there to be a way for a
> kernel space test suite to ask for kmalloc failures so that the failure
> paths can be tested.

Hi Matthew,

kmalloc fault injection is already implemented with
CONFIG_FAULT_INJECTION. For original fault injection, you more-or-less
ask to fail X% of allocations at random. It's great for testing
servers for stability, but not so well suited for testing. Recently
I've added so-called "systematic" fault injection
(/proc/thread-self/fail-nth) which is perfect for unit testing. It
allows to ask to fail N-th allocation request in the current task. So
a unit test for a syscall can do:

for (i = 0;; i++) {
  write(/proc/thread-self/fail-nth, i);
  syscall_under_test();
  if (read(/proc/thread-self/fail-nth) != 0)
    break;
}

which allows to examine each failure site one-by-one systematically
(without making random processes on the machine fail too).


> I think the idea of building parts of the core kernel libraries in
> userspace and testing them there has greater generality than just the
> IDA, IDR, XArray & radix tree and might profitably be adopted by other
> parts of lib/.  The userspace simulation of other parts of the kernel
> may well need to be extended.

This would be great. Besides providing faster turn-around time, this
allow us to finally have something like:

make test [subsystem]

which will run all tests for the subsystem locally (in parallel,
giving final OK/FAIL). This is not possible to in-kernel tests,
because they require a machine and an image, and it's not possible to
support everything that people use.