Usable Security

Notes:

In the design of the web whenever there was a trade-off between usability and security, usability always won.  Worse, those raising the usability issues were often not usability experts, they were just using it as a wedge to get what they wanted.

Usable security should be considered as part of Total Cost of Ownership.  In a typical system the licensing costs are dwarfed by the TCoO, which includes training, maintenance, and interoperability while constraining adaptability.  A truly usable system would have a large impact on the TCoO.

It is especially common to externalize security costs by dumping them on the user.

The Administrators are often, in a sense, “the enemy of usability”.  It is not necessarily in the interest of the Administrators and IT staff to push for especially usable systems because their job security is most stable when the end users are clueless and in need of constant help.  The more helpless the users, the more valuable those that assist them and provide support.

How would common metrics of usability help the problem?  Some of the most elusive metrics are trust metrics.  It is very hard to make a universal usability metric because oftentimes products will be perfectly usable in one way but not in another way.  For instance, if I download a file from iTunes it is perfectly usable in the ways Apple endorses, but not in the general sense. 

So what does it mean for a system to be “usable”?  Does context matter?  Who defines and sets the context?  Are the contexts exclusively set by the producer, or does the user have some say?

Metrics could be established based on dollar value lost by employee’s lost time.  As an example, in many cases accessing corporate resources through the firewall is so painful that many workers just choose not to remotely login, which has a direct impact on work that would otherwise be more readily completed.

Simply saying to estimate risk and the potential of vulnerability is tempting, but these are so very subjective that they’ll tend to vary widely and likely will continue to do so. 

There is no good, generally accepted taxonomy for discussing security risks.

There is often a temptation to model and base risk assessment on worse case scenarios.

There seems to be an implicit assumption that it is valuable to squash bugs.  But in some cases it may well be that the company benefits by leaving the bugs in, because doing so enables them to sell patches and perhaps helps sell the next version of the product.  Having dealt with the bugs, the company may well feel invested in the product and loyalty to the producer for fixing them.  There are real costs, both psychological and direct, that are incurred by fixing bugs.

Costs keep coming up in a negative sense.  How do you frame security and usability as a value-added rather than a cost avoided?  Which is more motivating?  Some members reported getting much more traction with the positive approach than the more typical negative approach.

More and more systems seem to offer insurance against incurred losses.  Does these programs really affect purchase decisions?  What kind of compensation is realizable?  How does one distinguish between a security incident and an availability incident?

How much should humans be included in the loop?  It is tempting to take humans out in many cases because their reactions are uninformed and can exacerbate the problem; in addition, many users would rather not have to make security decisions since they wouldn’t know how to respond anyways.  On the other hand, there is value in direct communication among security professionals in responding to many types of problems.

Trying to immediately respond to everything may overload internal resources and cause a huge increase in complexity.  The Roman Empire succeeded by responding precisely rather than immediately with more of a focus on containing system effects rather than maintaining strict perimeter integrity.  Such a model could prove very valuable to the security industry.

SOUPS included four parallel track discussion forums:
http://cups.cs.cmu.edu/soups/2008/program.html#discuss

Understanding PCI Regulations and Applying Strategies to Ensure Cardholder Privacy
Moderator: Eric Offenberg, IBM

Discussion topics will include:

* Understanding how safeguarding customer data protects a company’s bottom line
* Assessing the impact of PCI requirements on retailers, merchants, banks, and other affected corporations.
* Overcoming the fears associated with implementing technologies to become/remain compliant with PCI
* Discovering how PCI compliance can be leveraged to reduce costs and improve operational efficiency

Metrics for Characterizing Research Participants’ Technical Knowledge
Moderators: Serge Egelman and Ponnurangam Kumaraguru, Carnegie Mellon University

User studies can only contribute to human knowledge if they are generalizable across a known population.  Thus, the sample for a given user study needs to be describable so that it can be generalized to a larger population.  In many user studies, a user’s technical prowess can have a profound impact on the results of the study.  The ability to quantify (or at least classify) a user’s technical knowledge is becoming increasingly necessary in order to generalize studies across populations as well as compare the results of one study to another.  Some examples that researchers have used in the past are: (1) Educational background, (2) Internet usage, (3) Computer usage, and (4) Security knowledge.  But, these metrics are not consistently used in all the studies.  In this discussion session we plan to examine various metrics that can be used to quantify or classify technical knowledge.  We plan to present the metrics that have been used in previous studies and plan to get some consensus on the metrics during the session.

HCI-SEC Research, Private Data, and complying with the Common Rule
Moderator: Simson Garfinkel, Naval Postgraduate School

Even if you aren’t working with living breathing human subjects, your work into security and usability could easily require that you involve your organization’s Institutional Review Board (IRB).  That’s because 45 CFR 46, the Common Rule, covers not just the use of humans in experimental research but the use of data generated by humans under many circumstances.  In this discussion we will explore some of the ways that federal regulations may read on research, alternative interpretations, and formulate an agenda for change.

Balancing Security, Usability and Cost
Moderator: Ehab Al-Shaer, DePaul University

The main objective of deploying security in IT networks is to minimize risks of compromising or interrupting network services.  Due to lack of theoretical foundations, experimentations in this area, achieving cost-effective security configuration is very challenging.  As a result, most of the existing practice by even expert IT administrator is ad hoc, causing errors and instability.  This panel will address many of important related issues and others in this area:

* What are the main factors of security risk?  How to define and measure them objectively?
* How to consider exiting counter measures in estimating residual risk?  - Although performance is well-defined, defining metrics for flexibility and cost are not far from being realized.
* How to optimize security configuration to achieve balanced cost-effective security?  Is there a scalability issues here?  - How security configuration can be automatically optimized to track dynamic changing in risk?
* How a solution will be envision in a multi-domain networks where they often have conflicting objectives and independent administration?
* How this framework will be interfaced to end-user to enable easy to use and manage? 

http://cups.cs.cmu.edu/soups/2008/proceedings/p85Brustoloni.pdf

Electronic collaboration can greatly increase productivity, but there is a risk of liability for information misuse.  The current best practices are to use NDAs, but this can be cumbersome and many potential collaborations just never happen.

The researchers propose that Usage Controls (i.e.  Digital Rights Management) may make collaboration easier and more productive and may even remove the need for explicit NDAs.  The problem is that existing software-based DRM aren’t very reliable.  To overcome many of these shortcomings they have developed a Linux Security Module called UCLinux.  This paper examines the system and its interfaces.

TPMs (Trusted Platform Modules) are standardized and increasingly common in commercial computers.  Each component in the boot sequence measures integrity of the next component and extends results into a TPM PCR (platform configuration register).  The computer can verify itself to a remote computer by signing a challenge nonce and PCR values.  UCLinux is driven by the UCFS (Usage Control File System).  The UCFS is in an encrypted filesystem with a secret key based on the PCR value, thus insuring it will only load on an unmodified system.

They also designed a UI in which policies could be set when creating a file and in which the user would be prompted to accept usage restrictions when acquiring or opening a file.  They conducted a user study in which users role-played as an engineer making a design decision based on usage-controlled files retrieved from the Web.  In the first scenario there were no usage controls, in the second they were included.  There were seven documents available, four with acceptable usage policies and three without.  The study included ten participants.  For documents with acceptable policies there was no difference in document download between sessions.  For the documents with usage controls there was a notable reduction in downloaded documents in the session including usage controls.  In general the users found the system usable.

http://cups.cs.cmu.edu/soups/2008/proceedings/p77Inglesant.pdf

SOUPS 2008 Best Paper Award

In this paper the researchers explored how to make it so that non-security specialists are able to express access control rules in formal policy terms.  This is especially important because often people know what rules they want, but doesn’t know how to express them.

Access control is the ability to permit or deny the user of a particular resource by a particular entity.

The research was conducted using PERMIS, which makes role-based access control decisions as defined by policies expressed in XML.  The XML is editable using the GUI PERMIS Editor.  Permissions not explicitly granted are implicitly denied, so all statements are positive.

The approach used in this study was to provide a controlled natural language.  For the first phase of the study the researchers conducted interviews and foxu groups over 45 resource owners asking how they think about authorization requirements and how they express them.  For the second phrase they designed an ontology and control language driven by this data.

The interface displayed provides examples policies on the right side of the interface with an open text box on the left side of the interface.  When the user is happy with their policy they click Convert and can scan the Log panel which runs along the bottom of the interface for errors.  They tested this interface with 17 target users carrying out a series of scenarios.  They analyzed users going through the studies looking at time to complete tasks, whether users understand the building blocks, and whether they were able to successfully construct workable policies.

They found that users were not daunted by the controlled natural language, though the time taken and attempts per task were higher than they would have liked.  However, they believe that over time users would improve at using the interface.  They also found that they largely overcame the concern of users trying to write negative (”deny-based”) rules.

http://cups.cs.cmu.edu/soups/2008/proceedings/p65Vaniea.pdf

Websites tend to have an external privacy policy and the internal implementation of that policy.

The researches have continued their long-running work on SPARCLE, tool to help author and create policies that are both human and machine readable.  This talk is a review and expansion of the features of the Author Policy interface.

The framework allows the user to create policy rules using checkboxes and form entries, or using natural language, and is able to switch back and forth at any time.

Learning to write rules that the parser will understand is a bit of a challenge.  Part of the purpose in the tool is to help users understand the quirks of the parser and adjust their style so that the parser will work correctly.  They expanded the tool so that it shows the parse results in realtime.  Their interface shows how the input statement is parsed using coloring coding.

The researchers conducted a user study in which users were tasked with completing some rule modification and rule creation.  The control group did not get to use the new interface.  Results showed that the experimental group obviously liked and were using the new interface, even showing preference for the natural language interface over the structured interface.  However, it was also found that rule accuracy was roughly equal between the groups.  Interestingly, the groups made different types of errors; the control was making errors related to not making the proper changes in the first place, whereas those in the experimental group tended to make changes and have errors in the results.

The experimental group had trouble noticing when parsing results identified significant issues and were often surprised when researchers pointed out errors in the debriefing.  The control group tended to start in the Author interface and then review in the Transform interface.  The experimental group tended to just stay in the Author interface.  It was also found that the experimental group tended to feel continuously interrupted by the realtime parsing feedback.

Panel Moderator: Mary Ellen Zurko, IBM

Panelists:

* Stuart Schechter, Microsoft
* Phil Hallam-Baker, Verisign
* Jon Callas, PGP
* Tyler Close, HP

The panel started by pointing to Usability Evaluation Considered Harmful, which claims:

- a combination of methods triangulates and enriches discussion of a system’s validity
- evaluation done too early can kill promising ideas
- evaluation ignores cultural adoption and use

Is usable security different? 

Do products with features whose job it is to provide protection against maliciousness fare any differently?

Where are the appropriate ways to validate one’s work when the work is usable security?

When the goal is something other than published research, such as products or standards, how do the validation methods change?

Stuart Schechter, Microsoft

Position
Security assumptions that rest on user behavior must be scientifically tested.
Evaluations must not underestimate the adversary.

Usability problems are often the result of failures in the architecture or ecosystem built around a product.

He used Code Signing as an example.  While the Publisher is verified in the signed certificate, the Publisher is free to set the Product Name for the confirmation dialog to whatever they want.  Microsoft’s security assumptions in code signing were that users will avoid Products from Publishers they don’t trust.  This could have easily been tested to determine whether users were more swayed by the Publisher name or the Product name.

If the claim is something is not testable then there is implicit risk of mis-assumption.  If the known best is known to be poor then it could be worth taking the risk, but there is a solid risk.

Jon Callas, PGP

Position:

Designers know best.

Testing has its place.  It can tell you what color the buttons need to be, where to put them, and reveal user preferences.  Testing can reveal bugs.  It can kill bad ideas.  But, it cannot tell you what to do right.

Testing can impede progress.  Every user in the world knows that change is bad.  People like what they are used to.  They forget how much they hated what they have now (e.g.  initial complaints upon the original release of the current version). 

If training is required and something is truly new then testing will generally show that it sux.  Usable security requires change.  Design is an art form.  And like in art, “plot selected by focus group” does not bode well.

Sometimes when somebody has an idea that does indeed test well, you still need to be willing to say no.  This is especially true in security.  HCISec is hard.  But, we’re designers for a reason.  Any coherent vision is better than no vision at all.  Testing can help refine coherency and can point out gross deficiencies.

But don’t forget - everything we love now once sucked.

Phil Hallam-Baker, Verisign

Position:

The situation regarding Internet crime is bad.  We don’t have to evaluate the products available for protecting us from Internet crime - they all suck.  The situation may be bad, but experiments are making them worse.

Lab studies measure the wrong thing.  They don’t tell you how the user will use the product in practice - they’ll just be confused by the novel UI or whatever and all you discover is that users are confused by new things.  The mere fact that there is a discipline based around experiments does not mean those experiments are useful.

Scientists explore the world as it is.  Engineers want to change it.  Engineers require a map.  The map we have established from all our experiments is effectively a swath of Here There Be Dragons.  Cathedrals were built by trial - if it collapsed, it wasn’t a good design.  It was possible to build solid cathedrals by the 1400s - even though we couldn’t scientifically demonstrate the safety or concepts until the 1800s.

Design Law #0: If its not safe, its not usable.

“We can’t make it more secure because the user won’t understand it.  It won’t be usable.” Wrong.  If it isn’t safe, it isn’t usable.

Design Law #1: If the user cannot distinguish an unsafe action from a safe one, it is unsafe.

Tyler Close, HP

Position:

“Considered harmful” is a famous phrase in CS coined by Dijkstra.  This is effectively a usable security argument - the user can only internalize so much context.

What might Dijkstra say about usable security were he here today?

Dijkstra worked out from first principles the cognitive load necessary to handle the goto statement and determined that it must go away, even though he didn’t yet know how to replace it.

Correspondingly, experts should perhaps design development tools such that they can more easily create more usable software in the normal flow of development.  Won’t work you say?  But it has - in programming language design.  Is usability all that different?

Look at JavaScript - the programmers have a lack of understanding of the underlying syntax and execution model…but it doesn’t matter!  If only they could similarly make usable UI without having to understand the underlying model.  Dijkstra might buy this, but could well claim that UIs are just different.  The history of programming languages is a forward march of enabling the programming to express his/her intent. 

The design of a good programming language is at least as hard as the design of a good UI.

The way forward is expert review of the tools programmers use.  What sorts of changes might this entail?  The OK button is an awful lot like the goto statement.  The application is basically asking the user to take responsibility of the whatever happens next.  What would happen if we removed the control-flow construct of prompting the user in this way?  Sound painful?  Well, that’s what some said about goto.

http://cups.cs.cmu.edu/soups/2008/proceedings/p56Saxena.pdf

This research explored how to bootstrap a secure communication channel between two wireless devices when they have no prior association and no trusted third party.  Examples are pairing a WLAN laptop to an access point, or a Bluetooth cellphone and headset.

The proposal is to use an Out-Of-Band channel between the devices created with human perceptible (audio, visual, or tactile) output.  The OOB channel is “physically authenticatable”.  This limits the capabilities of attackers while striving for a minimal burden on usability.  The security model explored assumes an insecure, high-bandwidth wireless channel, and an authenticable, low bandwidth OOB channel.  The adversary has complete control over the wireless channel, and can eavesdrop on the OOB channel but cannot modify messages.

Prior work in the field can be found in a project called Seeing-Is-Believing.  Each phone displays a barcode is used to take a picture of the barcode off the other camera, thus ensuring mutual authentication.  This method is useful, but the barcode must encode 48-bits, meaning it isn’t easily adaptable to a more limited display.  And, of course, not all devices can be assumed to have an integrated camera.

A universal pairing method was proposed by Prasad-Saxena in which strings are display by both devices to the human user and the human verifies correctness.  This could be as simple as synchronously flashing LEDs or speaker tones.  But users can still make mistakes, especially in a distracting environment.

This research sought to improve on this by adding an ATD (Auxiliary Third Device) meant to validate the synchronous signaling and thus remove the possibility of human mistakes.  When performing automated pairing the user must
- press a button on one device to start pairing
- adjust the ATD’s inputs to focus on the devices being paired
- press a button to activate the ATD’s receivers
- press a button on one device to start SAS transmission
- accepting or rejecting the pairing session based on the ATD output

For testing, a laptop was used an an ATD with a 2.0M camera running at 30 FPS.  The devices being paired were simulated by LEDs on a breadboard.  They used a 15-bit SAS (Short Authenticated String) encoded over blinking LEDs.  They then did another test in which they synthesized the SAS to speech.

Both schemes were tested with 20 subjects manually confirming the outputs and using an ATD.  Each subject was presented with 24 test cases.  The goal was to determine if ATD did better than the users were able to on their own.  The results indicated that the ATD did make the process safer and less burdensome, though they found that the ATD made more errors for the case of audio output.  Whether the third device is a help or hindrance on speed of the operation depends on the capabilities of the ATD; in the case of the blinking LEDs the ATD was much faster, but was slower due to the relatively slower audio recoding process.

http://cups.cs.cmu.edu/soups/2008/proceedings/p35Hayashi.pdf

This research proposes a graphic login system in which the presented images at login time are highly distorted versions of the images chosen at password creation time.  The user should be able to recognize the distorted version of the picture they originally chose.  That said, there is a trade-off in that as distortion increases the ability to associate the original with the distorted copy may become more difficult for the user.

Unlike in many graphical password schemes, UYI allows users are allowed to provide the pictures since they’ll be distorted anyways.  The distortion used discard precise shapes and colors but preserves rough shapes and colors.  An attacker will then not be able to tell what the blurred images represent and won’t be able guess the proper image even if the user used highly personal pictures.

They completed research in which they asked users to distinguish the picture with different levels of distortion to find the region in which it would be very hard to recognize without knowing the original image but relatively easy to recognize when the image is known.  They implemented a prototype on the web and on a Nokia cell phone.

In the prototype the user creates their password by choosing three images.  They then proceed through a Practice session in which they have to choose their images from a grid with the proper image plus eight random incorrect images; during this phase there is explicit feedback on whether the user has chosen the proper image.  Finally, the user logs in during an authentication phase which prompts the user to select the proper image from a field including eight random images, but with no immediate feedback.  The user can choose the wrong image three three times before authentication is considered failed.

The researchers completed two usability tests.  The first had 45 participants and lasted 1 week; the second had 54 participants and lasted 4 weeks.  For the first study participants were divided into three groups with different degrees of image distortion.  In each group they were asked to choose three pictures to create the password, login on the first day, the second day, and one week later.  It was found that the non-distorted and somewhat distorted groups did about equal, with only the heavily distorted group having notable problems.

For the second test the 54 participants were divided into three groups.  This test was differentiated by having the user also log in again four weeks after the initial date, and by including a group in which the user’s were not able to provide their own photos and only ever saw the distorted images.  While the group with imposed, distorted images did the worst, the results were fairly similar across the three groups.

http://cups.cs.cmu.edu/soups/2008/proceedings/p44Yan.pdf

CAPTCHAs were originally invented at CMU.  The goal of a CAPTCHA is to allow humans through but block automated scripts.  They are now widely deployed as a method of preventing spam.

Text-based schemes typically require the use to complete a text recognition tasks.  Some sites offer a sound-based scheme, typically for accessibility reasons.  There have also been some image-based schemes, such as Microsoft’sw Assira.

This research is on designing usable and robust CAPTCHAs.  Many deployed CAPTCHAs are not very usable.  Many of them are also not very robust - low-cost attacks on schemes by Microsoft, Yahoo!  and Google will be presented at CCS’08.

A framework for CAPTCHA usability:
- distortion
- content
- presentation

It is well known that under distortion some characters as as 1 and l, O and 0, and 5 and S have a high potential for confusion.  In the name of security, Google and Yahoo!  CAPTCHAs have actually created new confusing characters, such as vv and w, cl and d, and rn and m.  It would found that ~6% of the CAPTCHAs received from Google included such confusing characters such that a normal user would be unable to distinguish the intended letters.  In the latest Yahoo!  scheme ~10% of the challenges include such confusing combinations.

An example of a content concern is whether the length of the CAPTCHA string is fixed or variable length.  A constant length is more usable, but is also easier for an attacker to segment the image since they know the proper number of segments. 

A place presentation comes up is in the use of colors in CAPTCHA design.  The researchers found that color can often be an impediment to usability, and sometimes even reduces security.  For example, many CAPTCHAs have a complexly colored background, but machines can usually distinguish the text from the colored noise and extract out the background, so it is has hurt usability without helping security.  In some cases color is overly relied upon because some CAPTCHAs don’t even distort the character, just relying on the confusion caused by colorful noise, but again, the noise can be stripped out relatively easily.

http://cups.cs.cmu.edu/soups/2008/proceedings/p24Dunphy.pdf

Passfaces is a commercial graphical password system where the password is a sequence of face images.  This leverages the fact that humans are typically rather good at facial recognition.  Another motivation of Passfaces is supposedly that it is hard to write down your password to share, but are they?  Often a single-sentence description seems to suffice.

The researchers conducted a study in which 18 participants were asked to describe 15 out of 45 faces (27 female & 18 male).  The average description took 23 seconds to write.  They found that female descriptions were typically longer than male descriptions.  Females used 654 distinct words while males used 567.  Hair was the most common thing described, followed by face shape, eyebrows, and nose.

They then had a completely separate group of users attempt to login to a mock passfaces system based on the descriptions provided in the first part of the study.  This included 56 participants under three conditions:

- random decoys
- visually similar decoys (8 alternate faces of the same sex as the target face)
- descriptively similar decoys (8 alternate faces similar looking to the target face)

In each condition the participant was given either five descriptions written by a male or five descriptions written by a female.  Across all trials, subjects were able to successfully login based on the descriptions 9% of the time (7 for random, 5 for visually similar, and 1 for descriptively similar).  The study found that the both the visually similar decoys and descriptively similar decoys were significantly better than the random condition, though neither method of choosing similar decoys was obviously better than the other.

http://cups.cs.cmu.edu/soups/2008/proceedings/p13Rabkin.pdf

Security questions aren’t always bad…though they often are.  But, the bad news is, they are getting worse.  A secret security question asks for a secret fact.  A personal security question asks about something meaningful to the user, but that they are willing to share.  Unfortunately, if users are willing to share this information in one context they may well be willing to share the same information in another context.

The researchers and volunteers went through the forgotten password mechanism at 20 banks and wrote down the steps in the authentication process, including a list of all accessible security questions.

It turns out most of the big banks and credit cards mostly don’t rely on personal security questions alone.  Many ask for SSN + acct number + PIN.  A few send e-mail messages.  Security questions were much more common with brokerages and on-line banks.

Different sorts of security weaknesses:
- guessable
- automatically attackable
- human attackable

The security of personal security questions are based on assumptions of information-retrieval hardness.  But information retrieval is improving rapidly.  Many answers can often be found, or deduced from information available on, a subject’s publicly available social networking page(s).  Many others can be found with a simple Google search, especially if the subject has a personal website.

Some suggestions to fix this would be to ask recognition-based questions rather than recall-based questions.  It may perhaps be useful to embed media into the questions.  Another approach would be to try different styles of questions; for example, the method detailed in Love and authentication asks the user to indicate Yes/No as to whether they like things in a list of topics where the authenticating subject has to match their initial answers.

http://cups.cs.cmu.edu/soups/2008/proceedings/p1Forget.pdf

The research explored a novel password selection strategy in which subjects would enter a password and have random characters shuffled in to add security to the password.  The researchers explored different methods of selecting and placing the characters.

The goal is not only to help users choose better passwords, but also to build off elements of Persuasive Technology to help them better understand what makes a password secure in hopes that the users will choose better passwords in general.

The research results found that those with shuffled passwords took longer to confirm their password initially, but did as well as the control when recalling them later.  The study had included a mental exercise between sessions to clear the subject’s memory to better simulate having a greater period between attempts.  They ran John the Ripper over the resulting passwords and were happy to find that not a single PTP-improved password was cracked!

Unfortunately, it was found that as users proceeded through the trial they tended to choose less secure initial passwords once they knew the PTP system would add more characters, perhaps limiting the total security gains realizable by PTP.