Message-ID: <OF4CB0DE09.E2E2433D-ON86256BFB.0073B75A@ey.com>
From: Ken.Williams at ey.com (Ken.Williams@...com)
Subject: Creating a publicly maintained vulnerability database

> Message: 6
> Date: Fri, 19 Jul 2002 16:42:13 -0400 (EDT)
> From: "Steven M. Christey" <coley@...us.mitre.org>
> To: full-disclosure@...ts.netsys.com
> Subject: [Full-Disclosure] Creating a publicly maintained vulnerability
database
> Reply-To: full-disclosure@...ts.netsys.com
>
> With organizations like CERT/CC projecting 4,000+ vulnerabilities this
> year alone, the amount of research and quality-checking that is
> required to maintain a good vulnerability database is growing
> prohibitive, even if most of the vulnerabilities were discovered and
> announced by many people in the community.

as usual, Steve hit the nail smack on the head.  back when i owned (and
almost
singlehandedly maintained) PacketStorm (PSS), i only had to deal with a few
hundred vulns/year.  i put in 80+ hour weeks, but i got it done (for the
most
part).  now, with the number of vulns reaching 4000+ per year, it really
does
take a team of highly skilled researchers to maintain a vuln db.  i know
exactly
what kind of people it takes, and i know just how interesting (and boring)
the
work can be, because i am currently spending about 180% (70+ hrs/wk) of my
salaried time maintaining vuln dbs and managing a team that researches and
validates vulns.

> As suggested by others, a publicly maintained vulnerability database
> is a possibility, but it would need a large-scale community effort to
> populate and maintain, and there would be issues of quality control.

maintenance and quality control become huge issues when you have to process
4000+ vulns/yr.  finding people who know what gdb is AND spell correctly
AND
can legally work in the US AND are willing to work for less than
$160,000/yr
AND show up for work every day AND can pass a simple background check can
also be a challenge.

> Maintaining a vulnerability database also requires some different
> skills than vulnerability research or system administration:
>
> - a stronger emphasis on writing for multiple audiences (technical
>   details and high-level summaries)

and if you want a really useful vuln db, you need both tech details and
hi-level summaries.

> - identifying different technical areas and finding/keeping skilled
>   people to cover them (e.g. crypto, Linux kernels, CGI, programming
>   languages, etc.)

and you have to have this too if you actually want the content in your vuln
db to be validated.  and what good is the vuln db if the content is NOT
validated???  i have yet to see a single attempt at a public vuln db that
contains VALIDATED CONTENT.  CVE comes closest (the content is very well
validated), but it is of course a vuln dictionary rather than a vuln
database.

> - defining what will be in the database (this was an issue in the
>   early days of CVE because everyone has different definitions of
>   "vulnerability," and it's still an issue to some degree)

and being the perfectionist that i am, i want ALL of it in the db.

> - ironing out details like workaround and fix information (even
>   determining whether the vendor has fixed the problem can be a
>   challenge; researcher-suggested patches can be broken; some
>   workarounds aren't feasible)

and if the vuln db is going to really be useful, you will make sure to have
all of the patch info, the vendor info, the 3rd party patches, and the
workarounds.  different people are going to need different solutions.

> - trying to distinguish between closely related vulnerabilities (is
>   there one vulnerability or two?  Did vendor X and Y really fix the
>   same issue?  Did vendor W's fix really address researcher Z's report
>   from two months earlier?)

stop it Steve!  my head already hurts without your mentioning this.

> - deciding on a severity metric (IMHO, high/medium/low must die)

and training all of the people who maintain your database to fully
understand and consistently apply that metric is another issue.

> - getting consistent terminology (your XSS is not my XSS (or CSS)!
>   Same with remote/local, directory traversal, etc.)
>
> - ensuring accuracy of information (which is sometimes problematic
>   even in "full disclosure" instances)

independent validation is the only good answer, and this will consume
89.54% of your time.

> - actually validating whether the reported vulnerabilities are real or
>   not (a daunting challenge for anything but the most commonly
>   deployed products and configurations)

especially tough if you are a small, public, non-profit org and you don't
have the $$$ to purchase the technology affected by the vuln (and the
vendor
won't give you a copy - not even an eval copy that may be different from
the
full licensed version mentioned in the original vuln advisory).

> - then designing, implementing, and maintaining the databases and the
>   server(s) that support it.

seems like many of the private, for-profit databases focus on this aspect,
at the expense of the content.  imo, the container is entirely useless if
the content isn't there.

> Chris Wysopal said:
>
> >Even so a completely non-corporate and free vuln database would be
> >something good for the community.
>
> NIST's ICAT database (http://icat.nist.gov) is freely available for

---[snipped stuff about icat and cerias]---

unfortunately, neither of those projects has offered a truly comprehensive
and timely vuln db yet.  until and unless some benevolent entity provides
big $$$ to fund such an endeavor, i doubt we'll be seeing a quality,
comprehensive (i want exploit code too!), *validated*, public vuln db any
time soon.  ICAT ain't bad though - need to be more timely and provide more
info for each vuln.

> Steve Christey
> CVE Editor

if i burn out and have to retire to the beaches of n. san fran, i will be
calling Steve and asking him to be my sponsor at Vulnerabilities Anonymous.

Regards,
kw

Ken Williams ; CISSP ; Technical Lead ; CVE Editorial Board
eSecurityOnline - an eSecurity Venture of Ernst & Young
ken.williams@...com ; www.esecurityonline.com ; 1-877-eSecurity



________________________________________________________________________
The information contained in this message may be privileged and confidential and protected from disclosure.  If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer.  Thank you.  Ernst & Young LLP

Powered by blists - more mailing lists