Designing and Evaluating Usable Security and Privacy Technology
There has been increasing interest in usable security and privacy
technologies, and studies that try to assess whether the technologies or
services meet usability and security requirements. As always in
multi-disciplinary research fields, there is a challenge of developing
experimental designs, methods and criteria that lead to valid answers.
Thus, the aims of the tutorial are
- To present a contextual approach to designing usable security and privacy
technologies and services,
- To review available experimental designs and methods for assessing
usability and security of such technologies and services, and
- To discuss how to apply these to gain valid results for research and
The first part of the tutorial will demonstrate how to apply the mantra
"know your users, their tasks, and the context of use" in the design of
potential security technologies in order to design with usability in mind. An
in-depth understanding of both usability and security requirements and
constraints must not only guide the developments of prototype or design
mock-ups and user scenarios, but also the questions to be addressed in the
evaluation, and how the evaluation is performed. The design of the evaluation
needs to consider the threats to internal and external validity of the data.
The tutorial will apply this to both research and commercial practice
We plan to use de-identified case study examples of mistakes made at each step
as a way of educating the participants and stimulating questions and
conversations about the material.
Researchers and practitioners involved in the design, development and
assessment of technologies or services that include a security or privacy
element – e.g. new authentication mechanisms, privacy-enhancing technologies,
policy authoring tools.
M. Angela Sasse is the Professor of Human-Centred Technology in the
Department of Computer Science at UCL. Since joining UCL in 1990, her research
has focused on shaping the design of emerging communication technologies and
services, particularly Internet-based ones. A key motivation of her research
is that new technologies should be “fit for purpose”, support and enhance
(individual and collective) human goals and activities, and provide a good
return on investment. This means investigating the performance of systems in
real operational contexts, and looking at the impact that a particular
technology has on individual and organizational users. Since 1996, she has
been developing a human-centred perspective on security, privacy, and trust.
Her early research on users’ problems with passwords (with Anne Adams) is one
of the most widely cited papers on usable security, and has been re-printed as
a “classic” in Cranor & Garfinkel’s Usability and Security. She
teaches a Masters-level course on People and Security at UCL and Oxford
University, and has supervised 6 PhDs on these topics as first supervisor
(Adams 2001, Brostoff 2004, Flechais 2005, Riegelsberger 2005, Weirich 2005,
Keval 2008). She has been Principal Investigator on over 20 research projects
– current long-term projects include Trust Economics, led by HP Labs,
and Privacy Value Networks, led by the Oxford Internet Institute. She
has (co)authored over 100 peer-reviewed publications (including 4-5 each in
ACM CHI, ACM Multimedia, International Journal of
Human-Computer Studies and the ACM New Security Paradigms
Workshop), and was a co-author of the Best Paper Award winner at SOUPS.
Clare-Marie Karat is a Research Staff Member in the Policy Lifecycle
Technologies department at the IBM TJ Watson Research Center in Hawthorne, NY.
Dr. Karat conducts HCI research in the areas of policy, privacy, security,
usability methods, and personalization. Dr. Karat leads the Server Privacy
Architecture and Capability Enablement (SPARCLE) Policy Workbench research
project to provide organizations and external users with the capability to
effectively manage the personal information held by organizations
(www.research.ibm.com/sparcle). She also has technical leadership roles in the
Army Research Laboratory International Technology Alliance (ARL ITA) project
on security policy management of information in mobile adhoc networks, IBM’s
Open Collaborative Research on Policy Frameworks for Security and Privacy
project with academic colleagues at CMU and Purdue Universities, and the
National Security Agency High Assurance Platform project to improve secure
information sharing. She has chaired international conferences and held a
variety of technical committee roles in the ACM CHI, HFES, IFIP INTERACT, and
SOUPS conferences. Dr. Karat has = presented keynote addresses, taught
seminars, published numerous papers in technical journals and conference
proceedings, and contributed to many books in the fields of HCI, policy,
privacy, security, and personalization.
Roy Maxion is on the faculty of the Computer Science and Machine
Learning Departments at Carnegie Mellon University. He is also director of the
CMU Dependable Systems Laboratory where the range of activities includes
computer security, biometric authentication, insider/masquerader detection,
and keystroke forensics in addition to general issues of hardware/software
system reliability and information assurance. A primary interest/concern is
the correctness and completeness of experimental methodologies. He teaches a
course on Research Methods for Experimental Computer Science. He has been
program chair of the International Conference on Dependable Systems and
Networks, member of the executive board of the IEEE Technical Committee on
Fault Tolerance, the United States Defense Science Board, the European
Commission AMBER advisory board, and other professional organizations. He has
consulted for the US Department of State as well as for numerous industry and
government bodies. He is on the editorial boards of the IEEE Transactions on
Dependable and Secure Computing, the IEEE Transactions on Information
Forensics and Security, and the International Journal of Security and
Networks. Dr. Maxion is a Fellow of the IEEE.
Think Evil (tm)
Tutorial Slides (PDF)
Security problems are different from every other problem in Computer
Science. Unlike the rest of the field, security is all about dealing
with an adversarial context: there exists opponents with means, motives,
and opportunities to disrupt the system. Thus when developing systems,
the first step is to understand the participants, including adversarial
participants, and be able to think like all sides in a conflict.
There exist both semi-formal models of adversarial decision making
("OODA loops") and informal techniques (thinking like your "evil twin")
that can help in guiding one's process when developing models of how and
why adversaries interact with the system. Likewise, attackers and
defenders can be constrained: only able to operate in specific ways, and
lines of attack can have endstates which may favor one side or another.
Finally, cost and motives can have huge impacts on outcomes.
This tutorial will focus both on the general theme of adversarial
thinking and real-world examples, including worms, botnet and viruses,
airport security, wall street, and personal financial security protocols.
Nicholas Weaver is a researcher at the International Computer Science
Institute in Berkeley, where he focuses on many issues involving network
security. One particular specialty is the network behavior of worms and
other internet-scale attacks, including understanding how fast worms can
spread, understanding the dynamics of previous network attacks, and
developing automatic network defenses.
Other areas have included both hardware acceleration and software
parallelization of network intrusion detection, defenses for DNS resolvers,
and tools for detecting ISP-introduced manipulations of a user's network
connection. He obtained his Ph.D. in 2003 from UC Berkeley, where he
focused on FPGA architecture, tools, and applications.
Eric Sachs - Redirects to login pages are bad, or are they?
| more information]
Many identity protocols rely on full-page redirects to pages that
may show login pages, such as SAML, OpenID, OAuth, Facebook Connect,
etc. There are many concerns with this approach in terms of
usability, as well as its potential to increase phishing. However
even though those concerns have been around for years, these redirect
based protocols are becoming much more common, and are supported by
large companies like Yahoo, Google, Facebook, Microsoft, AOL, MySpace
and others who care a lot about usability and phishing. So given
these potential concerns, why is the support for this approach
growing? In this presentation we'll cover a number of topics where
that industry has been learning some unexpected lessons:
- Browser autofill of password
- Users already logged in
- Full-page vs. hacked popup vs. optimized popup
- Admin auto-approved
- Phishing, malware, and password "confetti"
- % success rate of screen scraping vs. redirects
- % success rate of federated login vs. account creation
- Success rate of blocking screen scraping
- Protocol combos (oauth+openid/SAML, FB Connect)
- Explanatory text and power user options vs. simplicity
- Invitation flows
- Real world examples and stats
Product Manager, Google Security & Internal Systems. Eric Sachs has over 15 years of experience in the areas of user
identity & security for hosted web applications. During his 4 years
at Google he has worked as a Product Manager for many services
including the Google Account login system, Google Apps for your
Domain, orkut.com social network, Google Health, Internal Systems, &
Google Security. Currently Eric works on standards for data interoperability including OAuth, OpenID, and OpenSocial. He previously architected and led the Google Health interoperability initiative including work with industry efforts such as the Markle Foundation's Connecting for Health WorkGroup.
Prior to joining Google, Eric was CTO and co-founder of Interliant
which provided hosted corporate E-mail services. While at Interliant,
Eric led co-development projects with both IBM & Microsoft to build
platforms for hosting personalized web applications.
Eric Sachs graduated with a B.A. in computer science in 1993 from Rice University.
Threshold Things That Think: Usable Authoriaation for Resharing
Roel Peeters, Markulf Kohlweiss, Bart Preneel,
and Nicky Sulmon
Not One Click for Security?
Alan Karp, Marc Stiegler and Tyler Close
Privacy Stories: Confidence in Privacy Behaviors through End User Programming
Luke Church, Jonathan Anderson, Joesph Bonneau and Frank Stajano
A new graphical password scheme against spyware by using CAPTCHA
Haichang Gao and Xiyang Liu
The Impact of Expressiveness on the Effectiveness of Privacy Mechanisms for Location-Sharing
Michael Benisch, Patrick Gage Kelley, Norman Sadeh, Tuomas Sandholm, Janice Tsai, Lorrie Faith Cranor and Paul Hankes Drielsma
Designing for Different Levels of Social Inference Risk
Sara Motahari, Sotirios Ziavras and Quentin
Integrating Usability and Accessibility in Information Assurance Education
Azene Zenebe, Claude Tuner, Jinjuan Feng, Jonathan Lazar and Mike O'Leary
Educated Guess on Graphical Authentication Schemes: Vulnerabilities and Countermeasures
Eiji Hayashi, Jason Hong and Nicolas Christin
BayeShield: Conversational Anti-Phishing User Interface
Peter Likarish, Don Dunbar, Juan Pablo Hourcade and Eunjin Jung
Recall-A-Story, a story-telling graphical password system
Yves Maetz, Stephane Onno and Olivier Heen
Escape From the Matrix: Lessons from a Case-Study in Access-Control Requirements
Kathi Fisler and Shriram Krishnamurthi
The Impact of Privacy Indicators on Search Engine Browsing Patterns
Janice Tsai, Serge Egelman, Lorrie Cranor and Alessandro Acquisti
Privacy Suites: Shared Privacy for Social Networks
Joseph Bonneau, Jonathan Anderson and Luke Church
Usable Deidentification of Sensitive Patient Care Data
Michael McQuaid, Kai Zheng, Nigel Melville and Lee Green
Eran Toch, Ramprasad Ravichandran, Lorrie Cranor, Paul Drielsma, Jason Hong, Patrick Kelley, Norman Sadeh and Janice Tsai
Studying Location Privacy in Mobile Applications:‘Predator vs. Prey’ probes
Keerthi Thomas, Clara Mancini, Lukasz Jedrzejczyk, Arosha K. Bandara, Adam Joinson, Blaine A. Price, Yvonne Rogers and Bashar Nuseibeh
Treat 'Em Like Other Devices: User Authentication of Multiple Personal RFID Tags
Nitesh Saxena, Md. Borhan Uddin and Jonathan Voris
Textured Agreements: Re-envisioning Electronic Consent
Matthew Kay and Michael Terry
A Multi-method Approach for User-centered Design of Identity Management Systems
Pooya Jaferian, David Botta, Kirstie Hawkey and Konstantin Beznosov
Posters Showcasing Usable Privacy and Security Papers Published in the Past Year at Other Conferences
flyByNight: Mitigating the Privacy Risks of Social Networking
Matthew Lucas and Nikita Borisov
Conditioned-safe Ceremonies and a User Study of an Application to Web Authentication
Chris Karlof, J.D. Tygar and David Wagner
Graphical Passwords as Browser Extension: Implementation and Usability Study
Kemal Bicakci, Mustafa Yuceel, Burak Erdeniz, Hakan Gurbaslar and Nart Bedin Atalay
It's no secret: Measuring the security and reliability of authentication via 'secret' questions
Stuart Schechter, A. J. Bernheim Brush and Serge Egelman
It's Not What You Know, But Who You Know: A social approach to last-resort authentication
Stuart Schechter, Serge Egelman and Robert W. Reeder
Robert W. Reeder, Patrick Gage Kelley, Aleecia M. McDonald and Lorrie Faith Cranor
Who's Viewed You? The Impact of Feedback in a Mobile Location-Sharing Application
Janice Tsai, Patrick Kelley, Paul Hankes Drielsma, Lorrie Cranor, Jason Hong and Norman Sadeh
New Directions in Multisensory Authentication
Madoka Hasegawa, Nicolas Christin and Eiji Hayashi
Machine Learning Attacks Against the Asirra CAPTCHA
A Comparative Study of Online Privacy Policies and Formats
Aleecia M. McDonald, Robert W. Reeder, Patrick
Gage Kelley and Lorrie Faith Cranor
Capturing Social Networking Privacy Preferences : Can Default Policies Help Alleviate Tradeoffs between Expressiveness and User Burden?
Ramprasad Ravichandran, Michael Benisch, Patrick
Gauge Kelley, and Norman Sadeh
Short and long term research suggestions for NSF and NIST
Moderator: Nancy Gillis, National Academy of Sciences
The Computer Science and Telecommunications Board (CSTB) of the National Academies is hosting a Usability, Security and Privacy workshop in Washington DC on July 21 and 22nd, focused on identifying new usable security and privacy research areas for the benefit of NSF and NIST. Participants in this session will brainstorm to identify new "out of the box" research areas or to expand upon the list of pre-identified research questions:
What metrics should we be using to measure usable security?
How can we collect data to apply these metrics? How do we know when we’ve got the appropriate data?
How do we conduct user studies that provide accurate measurements in real world or realistic laboratory conditions?
What is the unit of measurement of usable security?
How do we measure the ROI on usable security?
How might losses due to poor user design for security and privacy be
quantified? If yes, how might that information be used to improve
usability in support of security and privacy?
Are we ready for developing a "usable security" standard? How viable is it? What is required to develop one? Who would develop it?
3. Economic Incentives
How do people perceive the value of information? If individuals are
less motivated to protect "cheap" information versus "expensive"
information, can we create an associated security system?
We can apply human factors/cognitive science methods (e.g., usability analysis, cognitive task analysis, safety/error analysis, etc.) to issues and case studies. What might we learn from such applications?
What research is needed to identify relationships or interactions among variables (such as trust and privacy values) that lead to more complex influences on usable security compared to usability of traditional IT systems?
How can we get to the users conceptual models and pair those models with security models?
Ecological Validity in Studies of Security and Human Behaviour
Moderator: Andrew Patrick
Conducting research on human behaviour in a security context is hard, and it is often difficult to witness authentic behaviour in a laboratory environment. Ecological validity refers to the extent to which the results of a test or experiment can be applied to the real-life of the people being studied. Using a series of case studies from research on security-related behaviours, Dr. Patrick will lead a discussion about the nature of validity in research, the issues surrounding ecological validity, and research techniques that can be used to increase the validity of security studies.
Invisible HCI-SEC: Ways of re-architecting the operating system to
increase usability and security
Moderator: Simson Garfinkel, Naval Postgraduate School
Most work in the field of HCI-SEC has looked at ways of improving
the user interface to improve security. In this break-out discussion,
we will look for a different path -- zero-visibility, zero-interaction
changes to applications and operating systems that will have the
impact of increasing security and usability and the same time.
Examples include: modifying file erase commands so that erased files
are actually deleted; using cryptographic disk erasure, so that disks
can be "erased" instantly (by forgetting the key); providing for
automatic backup of critical files through cloud computing. In this break-out session, we will chart other ways that usability and security can be aligned at the system level.
Technology transfer of successful usable security research into
Moderator: Mary Ellen Zurko, IBM
Technology transfer in any area is a notoriously difficult problem. Yet it is also universally desirable. Researchers want to see their work deployed and used successfully. Products want to have leading edge features that give them a competitive edge. This discussion session will bring together the members of the SOUPS community interested or experienced in such matters. How do we know when our research is ready for technology transfer? What are the avenues? What has been done successfully? What has been tried and failed? We ask both researchers and product people to come with their stories, to begin to build up a corpus or oral history around this area.
The family and communication technologies
Moderator: Linda Little, PaCT Lab, Northumbria University, UK
New communication technologies are increasingly being used in family and social contexts to support and extend relationships. Yet the social aspects of these communication technologies and impact upon family life are often overlooked by researchers and designers keen to create task-based products. With this in mind there is a need to consider and focus on the social aspects of communication technologies within the family if we are to better understand how and why people are using and adapting communication technologies to suit their family and social lives. Of course, the family also plays an important role in how society functions; it acts as a primary source for the development of socialization skills and moral values. However, we need to acknowledge that decision making and value setting differ between family groups. Moreover, we must recognise that families are becoming more dispersed and consequently changing the ways they communicate. The family no longer refers solely to a core group of two parents and 2.4 children. Families are diverse both in their structure and function. Divorce, step-family relationships and multigenerational bonds are all altering familial structures. There is frequent speculation regarding the future of the family and that this leads to assumptions of a general deterioration in family bonds. This deterioration is regularly associated with the increased physical distance between family members. The further apart family members live, the greater the negative effect on any subsequent interactions. Questions naturally arise related to social and moral values, trust, privacy, disclosure, exclusion, status within the home and also the impact upon the home/work/leisure divide. This discussion will consider the issues of context, purpose and benefit to see if we can build up a richer, more detailed account of real technology usage and impact upon family life.
How does the emergence of reputation mechanisms affect the overall
trust formation mechanisms, implicit and explicit, in the online
Moderator: Kristiina Karvonen, Helsinki Institute for Information Technology (HIIT)
Reputation systems are used in internet services where users need to make trust decisions concerning people and data they do not know beforehand. Reputation guides users' decision making, and e.g. in eBay, high reputation can lead to price premiums. Users need to make their trust decisions based on any data available, e.g. object description, logos, user history, and so on. They also need to induce data reflecting other users' satisfaction with the other actors, such as number of actions, ratings and possible comments. Also, their own previous experience and personality strongly affect what it takes for them to trust a service or not.
Online trust formation has many ingredients and has been widely researched on from various viewpoints, including technical, legal, social, psychological and philosophical. What makes for a success story for a reputation mechanism, where security is a real issue? How to build interaction in such a way that it enhances the quality of trust decisions made? What is the relative value of personal experiences as compared with information gained from available reputation mechanisms for the trust formation process? Is there a way to gather the information about user opinions related to their trust to various objects and web resources and, at the same time, preserve user privacy?
SOUPS 2009 is sponsored by Carnegie Mellon CyLab and Google.