Panel: Usability of Security Software - Is Open Source a Positive, Negative, or Neutral Factor?

July 17, 2009 by Richard Conlan

Moderator: Luke Kowalski, Corporate UI Architect, Oracle
Stuart Schechter, Microsoft Research
David Recordon, Open Platforms Tech Lead, SixApart
David Sward, Director of User Centered Design, Symantec
Nancy Frishberg, User Experience Strategist and BayChi chair
Rashmi Sinha, CEO, SlideShare

The opening premise is that Open Source is a neutral factor on usability of security software.

One of the first myths of open source security is that transparency of the code renders the result more secure and usable.  Conversely, this level of exposure leaves it excused to unethical hackers.

There was some research that showed that over time the OpenBSD does indeed include fewer vulnerabilities over time.  Over a period of 8 years the rate was shown to have reduced by half.  It does improve.  Maybe that wouldn’t happen with closed source, but Windows Vista vs.  XP may exhibit roughly the same properties.

OAuth is like your valet key for the web.  Google, Yahoo!, etc., had designed their own distributed auth protocols and it was proving annoying for developers, so the community came together and developed a common protocol.  Then back in February, when Twitter started using OAuth for SSO, it was noticed that there was a security vulnerability in OAuth when used in this manner.  The OAuth community came together with a proposed fix and involve the stakeholders in solving the problem.

How the OAuth Security Battle Was Won, Open Web Style

Building on what David said, it is important that open source not only use open code, but open standards, because otherwise understanding and fixing it can be very difficult despite the openness.

There has recently been a shift from how vulnerable the code is to how vulnerable the user is.  Social engineering is emerging as a greater danger than concrete code deficiencies.  As such, whether the code itself is opened or closed is less important when it comes to the effective security.

One thing to consider is the scale of the project and the visibility.  Merely being open doesn’t say anything of the expertise of who is looking at the code.  Not all eyes are equal.  Take Debian, which had an error in its OpenSSL implmentation for years - how many people knew about and were exploiting it without reporting it?

Ka-Ping Yee had done an experiment where he planted a bug in 550 lines of code and then set experts loose on trying to find it, but they had trouble finding it.  Given that, what are the odds of finding an exploit injected into Firefox?  Ka-Ping’s response: Static analysis alone, human review alone, and automated testing alone, are all easy to defeat; however, defeating all three in concert is actually quite difficult.

Open could make a statement that open source software gets fixed frequently.  But is something going to be more usable because it can be changed more often, or does having such a flexible UI render it more or less confusing to the average user?  Is this better or worse than packaged software that at least ensures a static UI between releases?

It’s important to remember that grabbing a nightly build of Thunderbird is not an average user.  These projects actually have release schedules, official releases, and the like.

The Open Source community finds bugs in all sorts of manners, including those listed above, but many successful projects have dedicated developers.  But how many UX designers are out there working on Open Source projects?  It can be very difficult for a UX designer to get involved because the available infrastructure is generally aimed at code developers.  Often these projects start with the code, but what happens to end user analysis?  Who does these steps?

How do corporations participate in Open Source projects?  Sun Microsystems contributed many developers to the Gnome project, providing over 450 different manners of icons for different resolutions and sizes.  This seems like something unlikely to be done by a volunteer effort.  Google open sources many projects and does Google’s Summer of Code, released Android, and encourages employees to routinely contribute to countless Open Source project.  On the UI front there is also Season of Usability.  Facebook has engaged the OpenID community and shared lessons from Facebook Connect.  Even Microsoft has Open Source w/ Microsoft.  Most of the contributions to Open Office come from Sun and IBM.  Oracle will be contributing Btrfs to the Linux community.  Part of the success of Open Source has been driven by the involvement of traditional organizations.

Were there any papers at SOUPS from the open source community?  It doesn’t seem so.  It would be great if the open source community was performing and reporting on the basic research.

One thing big differentiation is the supporting of bug reporting.  Simson Garfinkel made the point that Apple is very responsive when he submits bugs, and tends to follow up and respond.  The results on Open Source projects are generally pathetic.  They respond okay to security bugs, but seem to entirely ignore usability bugs - UI tickets often get closed after two years of being ignored.

On the flipside, it is not uncommon for users to report having gotten Open Source bug reports addressed within hours, whereas the commercial software world is often comparatively slow to respond and get new versions out.  Developers in the Open Source world are forced to serve many roles because the projects generally have much less infrastructure.

But there are a lot of Open Source projects that DO have a public relations team, a legal team, technical writers, and other functions more commonly associated with commercial software.  Is one of the fundamental problems that the Open Source support tools are themselves unpleasant to use?  Somebody suggested SuggestionBox.

Part of the problem with Open Source is rampant brand fragmentation.  There are oodles of products with cute little names, oodles of non-profit organizations, lots of different metaphors and interaction styles and assumptions.  Generally speaking, the companies tend to do a better job of managing their brand and the user experience.

But aren’t there lots of companies too?  Why do we have Microsoft and Apple and Google and Yahoo!?  When you have a big company and you have standards and have to follow those standards it can often stifle innovation.  Maybe the milieu in the Open Source community is actually a strength!  It is very easy for an Open Source extension to try new ideas, including those that break existing modes and assumptions.  Even if only 1% of the ideas are great ideas, those ideas can quickly propagate and be adopted.

Very few Open Source companies are driven by a business model.  Most are volunteer efforts, so it is unfair to apply the same expectations to Open Source projects.  If you use Open Source and the UI is kinda clunky and you don’t like it…well…you’re not paying for it, so are you really setting the proper expectations? 

Yes, there are some Open Source projects that care about usability and some that do not, but this true for larger closed-source companies as well.  But another angle to think of this may well be large vs.  small - when an Open Source project is catering to a relatively small community it doesn’t necessarily make sense to devote oodles of effort to being generically usable.  For instance, a project aimed at highly technical users probably shouldn’t be wasting time trying to be usable by novices.

People seem to be more willing and interested in contributing their time to try and improve the product.  When somebody comes across a bug on Google’s site they don’t necessarily go out of their way to write up a list of grievances and send them to Google, whereas they might for an Open Source project because the underlying models of Open Source invites that, whereas the underlying model of a large company may not be perceived in that way.

Part of the value of Open Source projects lies in their ability to quickly adapt and try out new protocols.  Take OpenID as an example - an Open Source project can easily integrate it and even strip it out again if need be, whereas a large company may have a lot of bureaucracy and debate around adopting the new technology.  Is this good or bad for general usability?  Is flux in the UI overly confusing or a fertile avenue of innovation?

In many ways the Open Source community is a meritocracy.  Even when experts in the field try to contribute to projects, sometimes they are ignored because they aren’t well known in the community.  How do we bridge these gaps?

There is sense in both the Open Source and Closed Source communities that security bugs are very important and need to be addressed promptly.  Usability bugs, however, are routinely ignored on either front, even when they cause poor security in practice.  Why?  How do we fix this?  What should be done when an entire community makes such a fundamental assumption?

There is a huge opportunity for universities to encourage students to participate in Open Source development.  It is good for the community, good for its users, and good for the students’ resumes.

Closing remarks:

The lines between conventional development models and Open Source are indeed blending.  Conventional corporations are contributing more, while Open Source projects are gaining real-world adoption and commercial acceptance.

The collaboration between independent developers and companies often result in the most robust communities and projects.  Ultimately the software ecosystem needs both and benefits highly from the collaboration between the two.

Open Source vs.  Closed Source is roughly a wash when it comes to usability and security.  Both have pros and cons, and neither is a clear win.

Its very possible that we’re asking the wrong question.  The real concern is how this will drive commoditation of the industry.

The Open Source community has been found to be a ready and willing partner in conducting usability and security research.  There should be more outreach to that community along these lines.

A lot of value seems to come from the capacity of the Open Source community to question the usability and security of the Closed Source community, rapidly innovate upon perceived shortcomings, and leverage their adaptability to demonstrate the best path forward for all to follow.