linux-kernel - Re: [ANNOUNCE v2] ktest.pl: Easy and flexible testing script for Linux Kernel Developers (now with config bisect)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20101118142623.GB13933@redhat.com>
Date:	Thu, 18 Nov 2010 09:26:23 -0500
From:	Vivek Goyal <vgoyal@...hat.com>
To:	Steven Rostedt <rostedt@...dmis.org>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Ingo Molnar <mingo@...e.hu>,
	Thomas Gleixner <tglx@...utronix.de>,
	Peter Zijlstra <peterz@...radead.org>,
	Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
	Jon Masters <jcm@...hat.com>, corbet <corbet@....net>,
	Jake Edge <jake@....net>,
	Mauro Carvalho Chehab <mchehab@...radead.org>,
	lmr@...hat.com, Greg Kroah-Hartman <gregkh@...e.de>,
	autotest@...t.kernel.org, Clark Williams <williams@...hat.com>,
	Arnaldo Carvalho de Melo <acme@...radead.org>,
	Tim Bird <tim.bird@...sony.com>
Subject: Re: [ANNOUNCE v2] ktest.pl: Easy and flexible testing script for
 Linux Kernel Developers (now with config bisect)

On Thu, Nov 18, 2010 at 09:08:37AM -0500, Steven Rostedt wrote:
> Question:
> 
> Has anyone tried this out? Does anyone feel that I should get this ready
> to be included in 2.6.38 tools directory?
> 
> Just want to know if I should go ahead and prepare it, or if it will
> just be a waste of my time.

I had done some testing only on initial internal version but did not
get a chance to do more testing later.

But I believe it is worth to put effort to keep this script going. I am
especially interested in part where I can just automate compilation and
testing of a patch on a remote machine.

Thanks
Vivek

> 
> Thanks,
> 
> -- Steve
> 
> 
> On Mon, 2010-11-08 at 18:45 -0500, Steven Rostedt wrote:
> > I previously posted the announcement about ktest (and shamelessly
> > promoted it at both Kernel Summit and Linux Plumbers), but I just
> > updated it with a new feature.
> > 
> >   TEST_TYPE = config_bisect
> > 
> > (See http://marc.info/?l=linux-kernel&m=128829496215347 for original
> > post)
> > 
> > With the new config_bisect, the config is tested in halves. To handle
> > strange dependencies, the following is done.
> > 
> > First the "bad-config" needs to be prepared for the current working git
> > tree. As with all other tests, a "min-config" may be specified and that
> > will be override any config settings in the "bad-config". Since this may
> > produce new settings to be set (or removed) some preparation needs to be
> > done. This is done automatically with ktest:
> > 
> > 1) overrides all configs with what is in the MIN_CONFIG or ADD_CONFIG
> > settings.
> > 
> > 2) Runs the result through a make oldconfig to see if what may have been
> > added or removed (it reports these).
> > 
> > 3) creates what is about to be tested and again runs the make oldconfig
> > to see if anything else might have been removed by dependencies.
> > 
> > All of the above changes are reported by the tool.
> > 
> > 
> > Finally, it gets into the bisect. It starts with a list of configs to
> > test. It cuts it in half, and then creates a .config file, and runs that
> > through oldconfig. If none of the configs in the list are enabled
> > (because they all depend on configs in the other half) then the second
> > half of the list is tried. If there's still no configs set, there must
> > be a circular dependency and the test fails here (but this should never
> > happen).
> > 
> > It compiles, and performs the type of test specified by the
> > CONFIG_BISECT_TYPE option (build, boot or test). If it passes, then all
> > the configs that were enabled in the .config are removed from the
> > configs to test and added to the good configs to always enable. This
> > allows for configs in the other half to be enabled if they depended on
> > these configs.
> > 
> > If the test fails, then we have a new subset of configs that is known to
> > cause the failure. Any config that was not set in the .config for this
> > run is removed from the test and will not be enabled again.
> > 
> > This iterates until we have one config left and that config will be
> > reported.
> > 
> > Because of dependencies and selects, the config reported may not be the
> > only config to cause the error. If you find that you can still produce
> > an error after disabling the config it reports, then simply disable it
> > and run the bisect again. Most likely it will turn up the problem. I may
> > automate this part in the future.
> > 
> > Anyway, you can download the latest ktest from:
> > 
> > git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/ktest.git
> > 
> > I need to write up a man page and perhaps more documentation. The
> > sample.conf that comes in that repo has loads of information on how to
> > do the tests.
> > 
> > 
> > Have fun!
> > 
> > -- Steve
> > 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/