Usable Security

Simson Garfinkel asks:

How do we determine the proper dosage of intervention?

We don’t have to identify studies as security studies. Should we do a comparative test of consent forms to see if they affect the experiment?

Does it introduce bias to offer users increased payment to motivate them to avoid attacks?

Can we look at attacks as a learning experience?

I’ve been looking at security as a ‘good’ in economics terms. So far, I’ve come up with a bunch of characteristics. My main question is how the market for security works, and in this sense I’ve narrowed in on how a user tests a security ‘good’ which bears some parallels to your question of usability tests.

My ‘goods’ show these characteristics so far:

* a test of the product by a simulated threat is:
- expensive and/or destructive, and/or
- the results cannot be relied upon to predict defence against a real threat;
* The attacker is an active actor in the process:
- profits by success
- bypasses defences and avoids statistical boxing
- can be relied upon to be dishonest
* a test of the product by a real threat is:
- difficult to arrange,
- insufficiently frequent to facilitate statistical analysis,
- could be destructive, and
* the results cannot be relied upon to predict defence against any other real threat.
* An actual event is both
- destructive and costly, and
- designed to minimise response, neutralising aggressive defence tactics, and
- versatile, seeks to migrate.
* The result is a negative sum game, in that
- as long as the attacker’s profit exceeds their costs, it is worth doing,
- the costs to the victim will generally exceed the profits to the attacker, the difference being damage,
- may or may not be an insurance market to compensate.

Quite a challenge!

—– How can we conduct studies that test how users behave under attack, while achieving a proper balance of ethical considerations and validity of the results?

One approach would be to try a *natural experiment*.

There are a number of organizations who do business, even conduct transactions, using third party websites. (e.g. Capital 0ne, The Art Institute of Chicag0). It wouldn’t be too difficult to compare their existing business model to a phishing attack. Collecting data on the transactions that these operations successfully carry out might provide a decent measure of consumer (non)resistance to transactions with dubious security.

Another natural experiment would involve websites who process credit cards without Verisign or the like. Yes, a few still exist.