full-disclosure - Open Letter on the Interpretation of "Vulnerability Statistics"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Pine.GSO.4.51.0601061436280.9254@cairo.mitre.org>
Date: Fri Jan  6 20:16:50 2006
From: coley at linus.mitre.org (Steven M. Christey)
Subject: Open Letter on the Interpretation of
	"Vulnerability Statistics"

On Fri, 6 Jan 2006, Georgi Guninski wrote:

> hahaha:
> http://cve.mitre.org/about/
> A Dictionary, NOT a Database
> (note the CAPS)
> so which way is it "NOT" or "A database"?

Hi Georgi, I've missed you.

According to the definitions proposed by Brian Martin of OSVDB, CVE is in
fact a database - HOWEVER it is a highly specialized one intended for
correlation and comparison across multiple tools and products.  That said,
90% of its consumers do not use it for that reason.  The FAQ should
probably be rephrased a bit.

> > RVI sources collect unstructured vulnerability information from Raw
> > Sources.
>
> read: parasites cut and paste from people who can do things.

Actually, they frequently augment the original work, especially if it
suffers from the Four I's problem - inconsistent, inaccurate, incomplete,
and/or incomprehensible.  Well-researched advisories like yours are the
exception, not the rule.

Every "RVI" or, if you wish, "database" provides extra value beyond what
is originally published.  Raw sources include lots of poorly written or
inaccurate advisories without any vendor fix information.  RVIs sort
through the cruft and produce something that is more usable to the average
consumer, often conducting additional analysis or interacting with the
affected vendor.

The average consumer does not have the time or the expertise to sift
through hundreds of information pieces from dozens of sources.

> > - LACK OF COMPLETE CROSS-REFERENCING BETWEEN RVI SOURCES.
>
> read: coley does not like it that there is no officially recognized
> usa funded database (NOT a dictionary) to rule em all and manipulate
> statistics.

Of course statistics can be manipulated any way you want to.  But CVE is,
as far as I know, the only RVI that has attempted to document and publish
at least part of its editorial policy, in the form of its content
decisions - *and* those content decisions received heavy review and
feedback by members of the CVE Editorial Board.

CVE and, I believe, OSVDB would like to achieve complete
cross-referencing.  This is a laudable goal but more resource-intensive
than currently allowed.  Most other RVI's cannot do this because they
compete with each other.

I personally want solid, accurate, complete vulnerability information that
can be independently reviewed and replicated.  In the areas where most
researchers fail to do this, RVI sources can help.

- Steve