linux-ext4 - RE: [PATCH linux-kselftest/test v2] ext4: add kunit test for decoding extended timestamps

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ECADFF3FD767C149AD96A924E7EA6EAF977D01A2@USCULXMSG01.am.sony.com>
Date:   Fri, 18 Oct 2019 01:30:42 +0000
From:   <Tim.Bird@...y.com>
To:     <brendanhiggins@...gle.com>
CC:     <tytso@....edu>, <skhan@...uxfoundation.org>, <yzaikin@...gle.com>,
        <linux-kselftest@...r.kernel.org>, <linux-ext4@...r.kernel.org>,
        <adilger.kernel@...ger.ca>, <kunit-dev@...glegroups.com>
Subject: RE: [PATCH linux-kselftest/test v2] ext4: add kunit test for
 decoding extended timestamps



> -----Original Message-----
> From: Brendan Higgins 
> 
> On Thu, Oct 17, 2019 at 3:25 PM <Tim.Bird@...y.com> wrote:
> >
> > > -----Original Message-----
> > > From: Theodore Y. Ts'o on October 17, 2019 2:09 AM
> > >
> > > On Wed, Oct 16, 2019 at 05:26:29PM -0600, Shuah Khan wrote:
> > > >
> > > > I don't really buy the argument that unit tests should be deterministic
> > > > Possibly, but I would opt for having the ability to feed test data.
> > >
> > > I strongly believe that unit tests should be deterministic.
> > > Non-deterministic tests are essentially fuzz tests.  And fuzz tests
> > > should be different from unit tests.
> >
> > I'm not sure I have the entire context here, but I think deterministic
> > might not be the right word, or it might not capture the exact meaning
> > intended.
> >
> > I think there are multiple issues here:
> >  1. Does the test enclose all its data, including working data and expected
> results?
> > Or, does the test allow someone to provide working data?  This alternative
> > implies that either the some of testcases or the results might be different
> depending on
> > the data that is provided.  IMHO the test would be deterministic if it always
> produced
> > the same results based on the same data inputs.  And if the input data was
> deterministic.
> > I would call this a data-driven test.
> >
> > Since the results would be dependent on the data provided, the results
> > from tests using different data would not be comparable.  Essentially,
> > changing the input data changes the test so maybe it's best to consider
> > this a different test.  Like 'test-with-data-A' and 'test-with-data-B'.
> 
> That kind of sound like parameterized tests[1]; 

...

> 
> [1] https://dzone.com/articles/junit-parameterized-test

Both Iurii and you pointed me at parameterized tests.  This sounds like
what I am talking about.  We have a lot of infrastructure for these concepts in Fuego,
but it's called something different.

I'm not sure if I should feel smart for having developed some of these concepts
independently or dumb for not having known about this (apparently) standard
nomenclature. ;-)

Thanks for the reference!
 -- Tim