lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Sat, 13 May 2006 12:17:31 +0100 From: "David Litchfield" <davidl@...software.com> To: "Ferguson, Justin (IARC)" <FergusonJ@...doe.gov>, "'Adam Shostack'" <adam@...eport.org> Cc: full-disclosure@...ts.grok.org.uk, ntbugtraq@...tserv.ntbugtraq.com, bugtraq@...urityfocus.com, dbsec@...elists.org Subject: Re: How secure is software X? Hi Justin, > One thing you have to keep in mind is that a lot of things are incredibly > variable when dealing with this subject. For instance, suppose you want to > ensure that the URI in a web server is not overflowable. So you test with > something like > > GET /[AAAAAAAAA x 4096] HTTP/1.1 > Host: foobar.com > Connection: close > > This is all fine and well, unless at 8192 is where the overflow takes > place, > or if I can stick as many characters as I want in, so long as I am using > HTTP 1.1 and not HTTP 0.9, or if I am using HTTP/1.1 and Host doesn't > contain a 36 backslashes, et cetera. > > This is generally why fuzzing is mostly inconclusive because it often > misses > a lot of conditions and you have essentially assured nothing. Without > in-depth analysis of each software package you are basically pushing snake > oil. There are just far too many variables to really standardize such a > thing. There are a few things to remember: 1) There are still too many products that fall to simple fuzzing. Having a standard that employs fuzzing as part of it means that (hopefully!) vendors will develop at least to that level - this raises the bar so to speak. 2) Not all fuzzers are born equal. Having written a fair few in my time I do realize that condition based fuzzing is important. A very simple but quite common example, to add to the ones you given, is with SMTP fuzzing. Some overflows only trigger after an EHLO greeting but not after a HELO. A good fuzzer and a good fuzzing process should take into consideration as many conditions as possible. 3) Fuzzing would only be part of the standard to be proposed. There are code/assembly scanning tools which can be incorporated amongst other things. Cheers, David > > Best Regards, > > Justin Ferguson > Reverse Engineer > NNSA IARC > 702.942.2539 > > "It is a capital mistake to theorize before one has data. Insensibly one > begins to twist facts to suit theories, instead of theories to suit > facts." > -- Sir Arthur Conan Doyle > >> -----Original Message----- >> From: Adam Shostack [mailto:adam@...eport.org] >> Sent: Friday, May 12, 2006 11:35 AM >> To: David Litchfield >> Cc: bugtraq@...urityfocus.com; >> full-disclosure@...ts.grok.org.uk; >> ntbugtraq@...tserv.ntbugtraq.com; dbsec@...elists.org >> Subject: Re: How secure is software X? >> >> >> Hi David, >> >> Very briefly because I'm swamped today: Please consider >> bringing some of this to Metricon >> (https://securitymetrics.org/content/Wiki.jsp?page=Welcome) >> >> Also there's a project of US DHS/NIST and probably others >> called SAMATE Software Assurance Metrics and Tool Evaluation >> http://samate.nist.gov/index.php/Main_Page >> >> which might be of interest. >> >> Adam >> >> On Fri, May 12, 2006 at 02:59:17AM +0100, David Litchfield wrote: >> | How secure is software X? >> | >> | At least as secure as Vulnerability Assessment Assurance >> Level P; or Q >> | or >> | R. Well, that's what I think we should be able to say. What >> we need is an >> | open standard, that has been agreed upon by recognized >> experts, against >> | which the absence of software security vulnerability can be >> measured - >> | something which improves upon the failings of the Common >> Criteria. Let's >> | choose web server software as an example. When looking for >> flaws in a new >> | piece of web server software there are a bunch of well >> known checks that >> | one would throw at it first. Try directory traversal >> attacks and the >> | several variations. Try overflowing the request method, the >> URI, the query >> | string, the host header field and so on. Try cross site >> scripting attacks >> | in server error pages and file not found messages. As I >> said, there's a >> | bunch of checks and I've mentioned but a few. If these were >> all written >> | down and labelled with as a "standard" then one could say >> that web server >> | software X is at least as secure as the standard - >> providing of course the >> | server stands up. >> | >> | For products that are based upon RFCs it would be trivial >> to write a >> | simple >> | criteria that tests every aspect of the software as per the >> RFCs. This >> | would be called Vulnerability Assessment Assurance Level: >> Protocol. If a >> | bit of software was accredited at VAAL:Protocol then it >> would given a >> | level of assurance that it at least stood up to those attacks. >> | >> | Not all products are RFC compliant however. Sticking with >> web servers, >> | one >> | bit of software might have a bespoke request method of >> "FOOBAR". This opens >> | up a whole new attack surface that's not covered by the >> VAAL:Protocol >> | standard. There are two aspects to this. Anyone with a >> firewall capable of >> | blocking non-RFC compliant requests could configure it to >> do so - thus >> | closing off the attack surface - from the outside at least. >> As far as the >> | standards go however - you'd have to introduce criteria to >> cover that >> | specific functionality. And what about different >> application environments >> | running on top of the web server? And what about more >> complex products such >> | as database servers? I suppose at a minimum for DB software >> you could at >> | least have a standard that simply checks if the server >> falls to a long >> | username or password buffer overflow attempt and then fuzz >> SQL-92 language >> | elements. It certainly makes standardization much more >> difficult but I >> | think by no means impossible. >> | >> | Clearly, what is _easy_ is writing and agreeing upon a VAAL:Protocol >> | standard for many different types of servers. You could >> then be assured >> | that any server that passes is at least as secure as >> VAAL:Protocol and for >> | those looking for more "comfort" then they can at least >> block non-RFC >> | compliant traffic. >> | >> | Having had a chat with Steve Christey about this earlier >> today I know >> | there >> | are other people thinking along the same lines and I bet >> there are more >> | projects out there being worked on that are attempting to >> achieve the same >> | thing. If anyone is currently working on this stuff or >> would like to get >> | involved in thrashing out some ideas then please mail me - >> I'd love to hear >> | from you. >> | >> | Cheers, >> | David Litchfield >> | http://www.databasesecurity.com/ >> | http://www.ngssoftware.com/ >> > > _______________________________________________ Full-Disclosure - We believe in it. Charter: http://lists.grok.org.uk/full-disclosure-charter.html Hosted and sponsored by Secunia - http://secunia.com/
Powered by blists - more mailing lists