linux-kernel - Re: automated test? (was Re: Linux 2.6.17.7)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <200607261510.03098.adq_dvb@lidskialf.net>
Date:	Wed, 26 Jul 2006 15:10:02 +0100
From:	Andrew de Quincey <adq_dvb@...skialf.net>
To:	Adrian Bunk <bunk@...sta.de>
Cc:	David Lang <dlang@...italinsight.com>,
	Arnaud Patard <apatard@...driva.com>, Greg KH <gregkh@...e.de>,
	linux-kernel@...r.kernel.org, stable@...nel.org
Subject: Re: automated test? (was Re: Linux 2.6.17.7)

On Wednesday 26 July 2006 14:02, Adrian Bunk wrote:
> On Tue, Jul 25, 2006 at 09:47:43AM -0700, David Lang wrote:
> > On Tue, 25 Jul 2006, Andrew de Quincey wrote:
> > >On Tuesday 25 July 2006 10:55, Arnaud Patard wrote:
> > >>Greg KH <gregkh@...e.de> writes:
> > >>
> > >>Hi,
> > >>
> > >>>We (the -stable team) are announcing the release of the 2.6.17.7
> > >>> kernel.
> > >>
> > >>Sorry, but doesn't compile if DVB_BUDGET_AV is set :(
> > >>
> > >>>Andrew de Quincey:
> > >>>      v4l/dvb: Fix budget-av frontend detection
> > >
> > >In fact it is just this patch causing the problem:
> >
> > <SNIP>
> >
> > >Sorry, I had so much work going on in that area I must have diffed the
> > >wrong
> > >kernel when I created this patch. :(
> >
> > is it reasonable to have an aotomated test figure out what config options
> > are relavent to a patch (or patchset) and test compile all the
> > combinations to catch this sort of mistake?
>
> If you think about it, you'll notice it's definitely not reasonable:
>
> #include <linux/module.h> brings you a dependency on 5 config options.
> #include <linux/pci.h> brings you a dependency on 6 config options.
>
> By only including these two headers you are at 2048 combinations.
> The number of valid configurations will be lower, but 500 test compiles
> sound realistically.
>
> With have a dozen #include's you might need more than a million test
> compiles.
>
> With a dozen #include's, you might need a trilion [1] test compiles.
>
>
> Compile errors are quickly catched and don't cause any serious problem.
>
> What bothers me more is that noone tested this patch against the kernel
> it was applied against.
>
> The submitter didn't test it works (he didn't even test the compilation).

Yes I did - I didn't test the final generated patch unfortunately since I 
assumed it worked. The kernel I _meant_ to diff against worked perfectly :(
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/