lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Wed, 12 Sep 2012 19:20:36 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	Brad Figg <brad.figg@...onical.com>
CC:	linux-ext4@...r.kernel.org, xfs-oss <xfs@....sgi.com>
Subject: Re: Ubuntu Ext4 regression testing

On 9/12/12 6:15 PM, Brad Figg wrote:
> I'm going to be doing some new runs so anything I find will be reported.

Dave Chinner also pointed out that i.e.

http://kernel.ubuntu.com/beta/testing/test-results/statler.2012-09-11_22-42-47/xfstests/default/control

seems to redefine, re-group, exclude etc various tests, and is taking "intelligence" out of the test suite itself.

I'd be wary of that; xfstests is dynamic - things get fixed, tests get added, groups changed, etc.

If you hard code for example "this test is for xfs" somewhere else, you might miss updates which add coverage.

Another example :

	    #'197' : ['xfs'],# This test is only valid on 32 bit machines

but the test handles that gracefully:

bitsperlong=`src/feature -w`
if [ "$bitsperlong" -ne 32 ]; then
        _notrun "This test is only valid on 32 bit machines"
fi

In general any test should be runnable; it may then issue 'not run' for some reason or other, but there's no harm in it - not as much harm as skipping regression tests because some config file got out of date...

and:

	    #'275' : ['generic'] # ext4 fails

but I just fixed that one up, and it should pass now.  Who will update the 3rd party config?

Failing tests absolutely should be run as well.  That information is as valuable as passing tests.  The goal is getting a complete picture, not just a series of "pass" results.  :)

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists