Message-ID: <CAJVRA1TBQH-h6gBA986S_-XXJ+pu4=UTXnb144WFtq6mNaSoBw@mail.gmail.com>
Date: Fri, 18 Apr 2014 22:38:20 -0700
From: coderman <coderman@...il.com>
To: Full Disclosure <fulldisclosure@...lists.org>
Subject: [FD] no good signals in infosec

if your industry lacks meaningful measurements,
 is devoid of independent, accurate, assessments,
   your industry has no good signals.

---

"No college,huh?"
"How many PhD's do you have?"
 - someone selling security using credentials rather than capabilities
as a signal.

---

https://www.schneier.com/blog/archives/2007/04/a_security_mark.html

 A Security Market for Lemons

More than a year ago, I wrote about the increasing risks of data loss
because more and more data fits in smaller and smaller packages. Today
I use a 4-GB USB memory stick for backup while I am traveling. I like
the convenience, but if I lose the tiny thing I risk all my data.

Encryption is the obvious solution for this problem -- I use PGPdisk
-- but Secustick sounds even better: It automatically erases itself
after a set number of bad password attempts. The company makes a bunch
of other impressive claims: The product was commissioned, and
eventually approved, by the French intelligence service; it is used by
many militaries and banks; its technology is revolutionary.

Unfortunately, the only impressive aspect of Secustick is its hubris,
which was revealed when Tweakers.net completely broke its security.
There's no data self-destruct feature. The password protection can
easily be bypassed. The data isn't even encrypted. As a secure storage
device, Secustick is pretty useless.

On the surface, this is just another snake-oil security story. But
there's a deeper question: Why are there so many bad security products
out there? It's not just that designing good security is hard --
although it is -- and it's not just that anyone can design a security
product that he himself cannot break. Why do mediocre security
products beat the good ones in the marketplace?

In 1970, American economist George Akerlof wrote a paper called "The
Market for 'Lemons'" (abstract and article for pay here), which
established asymmetrical information theory. He eventually won a Nobel
Prize for his work, which looks at markets where the seller knows a
lot more about the product than the buyer.

Akerlof illustrated his ideas with a used car market. A used car
market includes both good cars and lousy ones (lemons). The seller
knows which is which, but the buyer can't tell the difference -- at
least until he's made his purchase. I'll spare you the math, but what
ends up happening is that the buyer bases his purchase price on the
value of a used car of average quality.

This means that the best cars don't get sold; their prices are too
high. Which means that the owners of these best cars don't put their
cars on the market. And then this starts spiraling. The removal of the
good cars from the market reduces the average price buyers are willing
to pay, and then the very good cars no longer sell, and disappear from
the market. And then the good cars, and so on until only the lemons
are left.

In a market where the seller has more information about the product
than the buyer, bad products can drive the good ones out of the
market.

The computer security market has a lot of the same characteristics of
Akerlof's lemons market. Take the market for encrypted USB memory
sticks. Several companies make encrypted USB drives -- Kingston
Technology sent me one in the mail a few days ago -- but even I
couldn't tell you if Kingston's offering is better than Secustick. Or
if it's better than any other encrypted USB drives. They use the same
encryption algorithms. They make the same security claims. And if I
can't tell the difference, most consumers won't be able to either.

Of course, it's more expensive to make an actually secure USB drive.
Good security design takes time, and necessarily means limiting
functionality. Good security testing takes even more time, especially
if the product is any good. This means the less-secure product will be
cheaper, sooner to market and have more features. In this market, the
more-secure USB drive is going to lose out.

I see this kind of thing happening over and over in computer security.
In the late 1980s and early 1990s, there were more than a hundred
competing firewall products. The few that "won" weren't the most
secure firewalls; they were the ones that were easy to set up, easy to
use and didn't annoy users too much. Because buyers couldn't base
their buying decision on the relative security merits, they based them
on these other criteria. The intrusion detection system, or IDS,
market evolved the same way, and before that the antivirus market. The
few products that succeeded weren't the most secure, because buyers
couldn't tell the difference.

How do you solve this? You need what economists call a "signal," a way
for buyers to tell the difference. Warranties are a common signal.
Alternatively, an independent auto mechanic can tell good cars from
lemons, and a buyer can hire his expertise. The Secustick story
demonstrates this. If there is a consumer advocate group that has the
expertise to evaluate different products, then the lemons can be
exposed.

Secustick, for one, seems to have been withdrawn from sale.

But security testing is both expensive and slow, and it just isn't
possible for an independent lab to test everything. Unfortunately, the
exposure of Secustick is an exception. It was a simple product, and
easily exposed once someone bothered to look. A complex software
product -- a firewall, an IDS -- is very hard to test well. And, of
course, by the time you have tested it, the vendor has a new version
on the market.

In reality, we have to rely on a variety of mediocre signals to
differentiate the good security products from the bad. Standardization
is one signal. The widely used AES encryption standard has reduced,
although not eliminated, the number of lousy encryption algorithms on
the market. Reputation is a more common signal; we choose security
products based on the reputation of the company selling them, the
reputation of some security wizard associated with them, magazine
reviews, recommendations from colleagues or general buzz in the media.

All these signals have their problems. Even product reviews, which
should be as comprehensive as the Tweakers' Secustick review, rarely
are. Many firewall comparison reviews focus on things the reviewers
can easily measure, like packets per second, rather than how secure
the products are. In IDS comparisons, you can find the same bogus
"number of signatures" comparison. Buyers lap that stuff up; in the
absence of deep understanding, they happily accept shallow data.

With so many mediocre security products on the market, and the
difficulty of coming up with a strong quality signal, vendors don't
have strong incentives to invest in developing good products. And the
vendors that do tend to die a quiet and lonely death.

_______________________________________________
Sent through the Full Disclosure mailing list
http://nmap.org/mailman/listinfo/fulldisclosure
Web Archives & RSS: http://seclists.org/fulldisclosure/

Powered by blists - more mailing lists