From: coley at linus.mitre.org (Steven M. Christey)
Subject: Re: Are the number of vulnerabilities going up? is Symantec counting wrong?

The number of publicly reported vulnerabilities is increasing
significantly, based on counts of most major databases or other
vulnerability information sources, including CERT/CC, ISS X-Force,
SecurityFocus, CVE, etc.

Some reasons for the increase may be:

  (a) the increasing variety of bugs being looked at (e.g. XSS, length
      field tampering, integer signedness, and so on)

  (b) an apparently increasing number of people looking for bugs

  (c) the increasing variety of software being audited (e.g. network
      clients, image/document formats)

  (d) an apparently greater tendency of some people to audit obscure
      software packages created by amateur programmers.  NOTE: this is
      not a criticism of those practices.


>but what is going on here, if I read the statistics at
>http://icat.nist.gov/icat.cfm?function=statistics
>
>
>so 1307 vulns for 2002, down from 1506 in 2001!
>

ICAT is based on CVE, which is not yet complete for 2002; very roughly
speaking, it's 1307 vulns through August 2002.

We have 500 or more vulns from 2002 in the works, to be released in
the next month or so.

CVE tries to strike a balance between timeliness and correctness.
Determining correctness is challenging when you have multiple reports
from different sources that may or may not be the same issue.  As I've
said at various times, we take extra care not to make any mistakes in
this area.

One benefit of the CVE approach is that we regularly find important
inconsistencies that appear to go unnoticed by everyone else.  The
cost is that we are not as immediately up-to-date as the commercial
databases are.  However, we've been concentrating on being more timely
for important issues, and with the increasing involvement by vendors
and researchers, the most important bugs are usually assigned CVE
identifiers very quickly.

Counting vulnerabilities in general is a very tricky business.  You
can't easily compare numbers across different databases.  For example,
CVE "counts" vulnerabilities slightly differently than Symantec, ISS,
Neohapsis/SANS, and CERT do.  These variations often occur due to the
presence (or absence) of particular types of information.  In general,
CVE "undercounts" relative to these other sources, in an effort to
support a consistent level of abstraction across all publicly reported
vulnerabilities, regardless of the amount of information available.

CVE's "content decision" approach is geared towards consistency,
repeatability, and providing a solid foundation for metrics; see "A
Progress Report on the CVE Initiative" at
http://cve.mitre.org/docs/docs2002/prog-rpt_06-02/ for more details.

Another reason why comparison across databases is difficult is the
fact that each database uses a slightly different set of information
sources, so no single database is absolutely complete.  Also, some
databases intentionally exclude some bugs due to editor-level
criteria, e.g. if it's obscure software that the database's consumers
will not care about, or the database owner does not believe the
problem is really a vulnerability (e.g. consider the recent iDEFENSE
advisory on plaintext credentials stored in memory of SSH client
processes).

An example of a publicly stated editorial decision occured when
Neohapsis declared its intentions to omit most XSS bugs from its
weekly Security Alert Consensus alerts.  Interestingly, they received
no complaints (insert retort here, xss-is-lame ;-)

- Steve

Powered by blists - more mailing lists