In Search of Evaluation
August 6, 2005 by PingIn a recent e-mail message, Ian Grigg wrote that security professionals often seek perfection whereas users typically deal in fuzzy probabilities and moderate risks.
I run into this conflict in perspective all the time, since I frequently alternate between talking with security folks and usability folks, and am constantly amazed at what is obvious to one group and not the other (or is “obvious” to both groups, yet conflicting).
There are good reasons behind both philosophies. Understanding their origins can help us address the valid concerns raised by both.
1Actually, this is only true if the flaw is in the TCB. Sadly, most software today (e.g. Windows) flagrantly violates the principle of least privilege so, in effect, everything is the TCB, and security has in fact tended to zero over time. There are a few cases where programmers have been careful to build systems out of defensively correct components (e.g. privilege-separated SSH), and this has helped to prevent security from tending to zero over time.
The fundamental thing that makes security different from other computer-related disciplines is the presence of the active adversary. In particular, attackers adapt to security measures, always looking for new attacks. What this means for the practitioner is that the security of any flawed program tends to zero over time.1 The tiniest flaw becomes a vulnerability as large as any other at the moment that exploit code is posted online. Given this assumption, the only way to evaluate the security of a system is to attack it. Staying ahead of the game is a matter of attacking our own systems more vigourously than the attackers do, then patching every flaw we find.
The fundamental thing that makes usability different from other computer-related disciplines is the complexity of human behaviour. What this means for the practitioner is that predictions and models of how people will use human-computer interfaces are frequently wrong. Consequently, the only way to evaluate the usability of a system is to test it with real users in a controlled experiment. Staying ahead of the game is a matter of designing, testing, re-designing, and iterating as much as possible until a particular design demonstrates success in practice with the real target audience in its real context.
When you compare the two perspectives like this, it becomes clear that each group’s methodology fails to address the other group’s primary concern. How can we evaluate the security of human-computer systems when the users, the software, and the attackers are all moving targets? The best I’ve been able to come up with so far is this:
Hold yearly competitive user studies in which teams compete to design secure user interfaces and develop attacks against other teams’ user interfaces. Evaluate the submissions by testing on a large group of users.
This would at least try to address the adaptive-adversary and human-complexity issues while providing for yearly iteration of designs. It’s a tall order, though. Do you have any ideas for ways to make it more practical, or for other hybrid evaluation methodologies?
August 7th, 2005 at 10:52
Do we want secure user interfaces, or simple ones?
I see much of the problem with phishing as being the banks sending dancing rabbits, which distract from the funky URL.
If you send a text note saying “Please visit us at your bookmarked URL,” you gain much security by eliminating noise.
This is, of course, over-simplified, but I believe is worth considering.