linux-ext4 - Re: [PATCH linux-kselftest/test v2] ext4: add kunit test for decoding extended timestamps

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 17 Oct 2019 08:08:33 -0400
From:   "Theodore Y. Ts'o" <tytso@....edu>
To:     Shuah Khan <skhan@...uxfoundation.org>
Cc:     Brendan Higgins <brendanhiggins@...gle.com>,
        Iurii Zaikin <yzaikin@...gle.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>, linux-ext4@...r.kernel.org,
        adilger.kernel@...ger.ca,
        KUnit Development <kunit-dev@...glegroups.com>
Subject: Re: [PATCH linux-kselftest/test v2] ext4: add kunit test for
 decoding extended timestamps

On Wed, Oct 16, 2019 at 05:26:29PM -0600, Shuah Khan wrote:
> 
> I don't really buy the argument that unit tests should be deterministic
> Possibly, but I would opt for having the ability to feed test data.

I strongly believe that unit tests should be deterministic.
Non-deterministic tests are essentially fuzz tests.  And fuzz tests
should be different from unit tests.

We want unit tests to run quickly.  Fuzz tests need to be run for a
large number of passes (perhaps hours) in order to be sure that we've
hit any possible bad cases.  We want to be able to easily bisect fuzz
tests --- preferably, automatically.  And any kind of flakey test is
hell to bisect.

It's bad enough when a test is flakey because of the underlying code.
But when a test is flakey because the test inputs are
non-deterministic, it's even worse.

						- Ted