linux-kernel - Re: automated test? (was Re: Linux 2.6.17.7)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20060725212001.GA5493@merlin.emma.line.org>
Date:	Tue, 25 Jul 2006 23:20:01 +0200
From:	Matthias Andree <matthias.andree@....de>
To:	Arjan van de Ven <arjan@...radead.org>
Cc:	David Lang <dlang@...italinsight.com>,
	Andrew de Quincey <adq_dvb@...skialf.net>,
	Arnaud Patard <apatard@...driva.com>, Greg KH <gregkh@...e.de>,
	linux-kernel@...r.kernel.org, stable@...nel.org
Subject: Re: automated test? (was Re: Linux 2.6.17.7)

On Tue, 25 Jul 2006, Arjan van de Ven wrote:

> well you can do such a thing withing statistical bounds; however... if
> the patch already is in -git (as is -stable policy normally).. it should
> have been found there already...

The sad facts I learned from Debian bug #212762 (not kernel related) that
culminated in CVE-2005-2335 (remote root exploit against older
fetchmail) and from various qmail bugs Guninski discovered:

- a bug need not necessarily be found soon after introduction

- a bug report may not convey the hint "look at this NOW, the shit
  already hit the fan"
  (sorry, I meant to write: look NOW, it's urgent and important)

- an automated test to catch non-trivial mistakes is non-trivial in
  itself, and - what I've seen with another project I was involved with,
  and more often than I found amusing - is that the test itself can be
  buggy causing bogus results.

That doesn't mean I object to automated tests, but "it should have been
found by now" (because the source is open, someone could have tested it,
whatever) just doesn't work.

-- 
Matthias Andree
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/