From: yossarian at planet.nl (yossarian)
Subject: Was: Full Disclosure = Exploit Release - No disclosure No Fix

> yossarian wrote:
> > What info do we need? I think their must be a reasonable amount of
> > intelligence here, maybe we can - like the IETF proposal I mentioned
> > earlier - combine some efforts, and get it over with. If we can settle
on
> > the major questions, like the one Paul Schmehl posted, we can try to get
> > some real answers.
> >
> > Making a list of several hundred unsolved issues where notification took
> > place, or fixes took ages, can be done in a few weeks. In risk
migitation, I
> > prefer to work with profiles - not only of attackers, but also of
vendors.
> > It should be possible to do in reality what Gartner did for show, in
2000,
> > when they said that by the end of 2002 MS would have become better at
> > writing secure IIS, probability 0.8. Sure it is a lot of work, and
certainly
> > classification of unfixed, late fix, feasibility of fix in set time etc
will
> > be a nightmare, and the risk of lawsuits is real, but the benefits would
be
> > great. For the bigger vendors, statistics will iron out mistakes - and
its
> > the trends that count. Maybe they are getting better at it, who knows?
And a
> > propos the lawsuits, we don't all live in the USA.

BB wrote
> Something like that has been at the top of my list for some time, when I
> get some free time and corresponding motivation. :)  Securityocus used to
> have a decent set of simple statistics for how many vulns there were per
> vendor per time period.  I wanted to add on to that the time to patch.
So,
> take all the vulns for a particular set of vendors for say 2002.  Look up
> when they were publicly announced.  Perhaps see if you can determine when
> they were discovered and reported, too.  Then see how long it took for the
> patch to be made available.  Microsoft wouldn't look too terrible under
> those circumstances, except for perhaps the unpatched IE bugs.  Sun would
> probably look horrible.

Maybe standardisation in disclosure would help the statistically inclined:

OK, so what would be useful:
date of discovery, date of 1st notification, date and summary of response,
date of fix release - if any, single point of security notification at
Vendor Y/N. This should be the bare minimum, I guess. Quality of fix would
be nice, since some fixes don't fix. Fix vs. workaround could be useful,
disabling a certain feature is IMHO no fix, even if the feature is generally
considered useless - it might be useful to some, if it ain't, remove it. I
don't think a server containing this info should be in the US, since it will
be damaging to certain commercial interests.

> That probably would help a lot in terms of pciking a vendor.  I don't know
> that it addresses the question of whether exploits should be released.
> Perhaps it says that it's unfair to release Solaris exploits? :)

Statistics usually shed some light on the whereabouts of the facts. It will
not give a direct answer as to the effectiveness of releasing sploits
directly, but what it can do, is making it more effective, since it will
single out bad vendors, and give them an opportunity to better themselves.
If is a benchmark, vendors want the highest grades - it should competative.
Make it grades, so the impact will be clear, even to sales and management
;--)

> Back to the original point... what kinds of things do people want to know
> in order to take one side or another on whether full-disclosure is a good
idea:

> -What do vendors do when they aren't forced to patch things by public
> disclosure?  We know what they did many years ago, but now that they "know
> better" what would they do if full-disclosure went away?

Hard to answer, these what if questions.
>
> -How often are vulnerabilities independently discovered/exploited/shared
by
> bad guys?  Define bad guys.  We only know of a few instances here and
> there, and have the bragging of some vocal blackhats that can't be
> considered particularly trustworthy.

Well, you'd have to monitor all covert channels on the web to find out.
Maybe Carnivore will help? Nah. Not realistic, i suppose.

> -How often did someone "need" someone else's script to break into a box?

With only a estimated meagre 2% of attacks detected, who can tell? Define
attack. And not all attacks are scriptable, only certain types - you can
only script for flaws in systems used, not for all possible network design
or system implementation errors. Could you imagine an midsize IT company
setting up VPN to its corporate network without enabling encryption - I have
seen it. Can you imagine a major Telco using open shares where any connected
user has admin - No you wouldn't but it has happened - and has not changed
yet. Would you write a script for that - unlikely. Competent people usually
cannot foresee nor understand the errors made by the others. Of course some
people need scripts, but stuff like nessus is hardly usefull unless you can
code your own script. At this moment, Nessus can test 1165 different vulns,
gathered over the years. I once did some research, and I found out that over
a period of 6 months, 588 new vulnerabilities were posted on major lists.
(april - sept 2001). So Nessus holds less than 1 years worth of vulns
announced. Of course these are testing scripts, not actual sploits, but
count the number of sploits on supposedly black hat servers. At most 200 per
year emerge. Many go for the same vulns, quite a few do not work under any
circumstances. Maybe the working ones are used to break hundreds of boxes,
who knows - but not likely. I think it is very hard to prove the relation
between available scripts - mostly just probing scripts - and actual
attacks. Unless you consider a probe an attack. If it is very hard to prove,
chances are (Occams Razor) that there is no relation. The only case you can
make, is for viruses, where toolboxes can empower the dim. But this is not
breaking into a box.

> -How would the patch rate of customers change with more or less
> disclosure/hype?

Would be Nice to know, but I think to hype does not help. Where do these
hypes happen - on dedicated lists. They are not read by the average admin.

To this list of unanswereable questions I could add the ratio of security
fixes with or without preceding full or half-full disclosure. BTW, whatever
happened to disclosing 'somewhere inbetween', where only skilled people
could understand the technical details of an advisory and turn it into a
script?

So I guess we by this venue can never prove if full disclosure is a good
idea. But maybe it is not the correct question - we want vendors to build
safer software, not prove we can find holes and quibble about the credits,
commercial interests etc. Full disclosure is just a means to an end, and we
cannot see the end getting any closer. Hence my suggestion of a benchmark,
available to all.

Some questions/issue's on my list for the benchmark.

How do we make it a benchmark (i.e. understandable for the many, and useable
for the consultant types). A software company, will it be judged on the
number of programs they sell, the number of lines of code (say security hole
per 1000 lines of code?), should the type of program be taken into account -
with a reverse bonus if a program does not need external communication but
still has remotely exploitable holes? (completely flawed versus just
security flawed).

Other probs

Researchers follow trends and find the type of vulns they are looking for.
Remember when embedded or default password were 'in'. Hardly see them
anymore - now it has to be buffer overflows, header manipulation or unicode
related. Does this mean the older types of flaws aren't there anymore? If
true, it would prove that the industry is getting better at securing their
products. If false, it just proves the few researchers find flaw any
direction they care to look at, and many types of vulns will pass completely
unnoticed, since they can only look in one direction at a time.

Another major security problem I have never seen adressed here, nor anywhere
else, is that the majority of people on the web is no longer native english
speaking. (US 40%, plus some in UK, Canada, Down Under and in India, might
just be half. Is this a security problem? I have noticed from reading
security lists in other languages, that sometimes the vulns are not at all
posted on the big, english forums. This will increase in the future. How to
explain a complex hole in some american software to a vendor, if you can
barely manage the language? They will not bother. So in the future, the
vendor notification will drop.

Research of few years ago (1998 if I am correct) proved that systems are
significantly less patched in countries were use of english is less common.
Server software, databases - many of the common programs or books are not
translated. Advisories are not translated. Bugtraq, this list and readme
files with updates and hotfixes are in english. I have done networks in
Italy and France - well, the software was in English but the admins could
rarely make a remotely understandable sentence, let alone notify a vender
(or read the manuals). When the software was in the native language, there
were no language specific patches, and the english patches didn't work
properly, or changed part of the language in the menu's to english - or
both. The security industry is also mainly american, or english-centered.
The SQL worm proved again that secured shops can be hurt by unpatched
systems elsewhere. Security has always been an uphill battle, but it is
getting steeper.

The security of the Net therefore will decrease, if vendors don't extend the
support to other languages, and independent books are not translated - since
it is not commercially viable. And the feasability of a complete list of
known exploits will also decline. If 1 million chinese know, but no one
cared to translate, does it qualify as a know exploit, or will it be a 0Day?

My guess is that the best thing security researchers can do for the long
term, is learn chinese. Especially if the corporate marketing strategy
prescribes postings vulns - translating is a lot easier than crunching code.

Powered by blists - more mailing lists