05-436 / 05-836 / 08-534 / 08-734 Usable Privacy and Security

Fall 2011: GHC 4102, Tuesdays and thursdays 3-4:20 pm
Class web site: http://cups.cs.cmu.edu/courses/ups-fa11/
Class mailing list: http://cups.cs.cmu.edu/mailman/listinfo/ups

Students in this course may also be interested in joining the CUPS mailing list.

This course does not use Blackboard.

Professor: Lorrie Cranor
Email: lorrie AT cmu DOT edu
Web: http://lorrie.cranor.org/
Phone: 412-268-7534
Office: CIC 2207
Office hours: Mondays 9:30-11:30 am and by appointment

Teaching assistant: Rich Shay
Email: rshay AT cmu DOT edu
Office hours: by appointment

Course Description

There is growing recognition that technology alone will not provide all of the solutions to security and privacy problems. Human factors play an important role in these areas, and it is important for security and privacy experts to have an understanding of how people will interact with the systems they develop. This course is designed to introduce students to a variety of usability and user interface problems related to privacy and security and to give them experience in designing studies aimed at helping to evaluate usability issues in security and privacy systems. The course is suitable both for students interested in privacy and security who would like to learn more about usability, as well as for students interested in usability who would like to learn more about security and privacy. Much of the course will be taught in a graduate seminar style in which all students will be expected to do a weekly reading assignment and each week different students will prepare a presentation for the class. Students will also work on a group project throughout the semester.

Readings

Readings will be assigned from the following texts (available in the CMU bookstore and from all the usual online stores). Additional readings will be assigned from papers available online or handed out in class.

Security and Usability: Designing Secure Systems that People Can Use, edited by Lorrie Cranor and Simson Garfinkel
Research Methods in Human-Computer Interaction. Lazar J, Feng J, Hochheiser H.

Additional readings will be assigned from the course reading list. Most of these readings are in papers available online. In cases where a subscription is required for access, access should be available for free when you are coming from a CMU IP address (on campus or via CMU VPN).

Course Schedule

Note, this is subject to change. The class web site will have the most up-to-date version of this calendar.

Date	Topics	Assignment To be done before coming to class
Tuesday, August 30	Course overview and introductions
Thursday, September 1	A Framework for Reasoning About the Human in the Loop [slides] Approaches to making security usable Human-in-the-loop framework Human threat identification and mitigation process	Required reading: L. Cranor. A Framework for Reasoning About the Human in the Loop. Usability, Psychology and Security 2008. W. K. Edwards, E. Shehan and J. Stoll, Security Automation Considered Harmful? Proceedings of the IEEE New Security Paradigms Workshop (NSPW 2007). Optional reading: Motivation, models, and approaches; Research Methods, Chapter 1 Introduction
Tuesday, September 6	Introduction to HCI methods and UI design [slides] Usability and user interfaces Design Prototyping Evaluation	Required reading: Security and Usability, Chapter 1 Psychological Acceptability Revisited (M. Bishop) Security and Usability, Chapter 2 The Case for Usable Security (M. A. Sasse and I. Flechais) Security and Usability, Chapter 3 Design for Usability (B. Tognazzini) Optional reading: Motivation, models, and approaches Bring completed presentation topic preference form to class
Thursday, September 8	Designing experiments [slides] Types of experiments Experimental design process	Required reading: Security and Usability, Chapter 4 Usability Design and Evaluation for Privacy and Security Solutions (C.M. Karat, C. Brodie, and J. Karat) Research Methods, Chapter 2 Experimental Research Research Methods, Chapter 3 Experimental Design Optional reading: HCI methods and experimental design Homework 1 due - Reading summaries from 9/1, 9/6 and 9/8; complete human subjects training and submit certificate [+ summary of 1 optional reading]
Tuesday, September 13	Surveys, interviews, and focus groups [slides]	Required reading: Y. Wang, S. Komanduri, P.G. Leon, G. Norcie, A. Acquisti, L.F. Cranor. I regretted the minute I pressed share: A Qualitative Study of Regrets on Facebook. SOUPS 2011. L. Little, E. Sillence and P. Briggs. Ubiquitous Systems and the Family: Thoughts about the Networked Home. SOUPS 2009. [see also the videos used in the study] Research Methods, Chapter 5 Surveys Research Methods, Chapter 8 Interviews and Focus Groups Optional reading: HCI methods and experimental design; Research Methods, Chapter 6 Diaries; Research Methods, Chapter 7 Case Studies
Thursday, September 15	Mechanical Turk studies	Required reading: R. Shay, S. Komanduri, P. Kelley, P. Leon, M. Mazurek, L. Bauer, N. Christin and L. Cranor. Encountering Stronger Password Requirements: User Attitudes and Behaviors. SOUPS 2010. S. Komanduri, R. Shay, P. Kelley, M. Mazurek, L. Bauer, N. Christin, L. Cranor, and S. Egelman. Of passwords and people: Measuring the effect of password-composition policies. CHI 2011. P.G. Kelley, L.J. Cesca, J. Bresee, and L.F. Cranor. Standardizing Privacy Notices: An Online Study of the Nutrition Label Approach. CHI2010. Kelley, P. G. Conducting usable privacy and security studies with Amazon's Mechanical Turk. In Proceedings of the USER Workshop at the Symposium on Usable Privacy and Security (SOUPS) (2010), USER 2010. Optional reading: HCI methods and experimental design Homework 2 due - Reading summaries from 9/13 and 9/15; conduct password survey with at least 8 Andrew account holders and enter data into class spreadsheet (links to survey and spreadsheet to be provided) [+summary of 1 optional reading]
Tuesday, September 20	Introduction to security [slides] Guest speaker: Lujo Bauer	Required reading: Security and Usability, Chapter 13 Goals and Strategies for Secure Interaction Design (K. Yee) Security and Usability, Chapter 34 Why Johnny Can't Encrypt (A. Whitten and J. D. Tygar) Research Methods, Chapter 4 Statistical Analysis Optional reading: Security and threat modeling
Thursday, September 22	Introduction to privacy [slides] Defining privacy Online privacy issues Privacy by design Discussion of Braunstein et al. 2011	Required reading: Security and Usability, Chapter 19 Privacy Issues and Human-Computer Interaction (M. Ackerman and S. Mainwaring) Security and Usability, Chapter 21 Five Pitfalls in the Design for Privacy (S. Lederer, J. Hong, A. Dey, and J. Landay) Security and Usability, Chapter 24 Informed Consent by Design (B. Friedman, P. Lin, and J. Miller) A. Braunstein, L. Granka, and J. Staddon. Indirect Content Privacy Surveys: Measuring Privacy Without Asking About It. SOUPS 2011. Optional reading: Privacy Homework 3 due - Reading summaries from 9/13 and 9/15; Analyze class password survey data (do not do entropy estimates) and compare these results with Shay et al 2010, discuss how well the class study replicated the original paper and what might account for any differences in results (counts as extra homework) [+ summary of 1 optional reading]
Tuesday, September 27	Observing users in the field [slides] Experience sampling Laboratory studies [slides] Go over project topics and distribute project preference forms	Required reading: A. De Luca, M. Langheinrich and H. Hussmann. Towards Understanding ATM Security - A Field Study of Real World ATM Use. SOUPS 2010. Consolvo, S., Smith, I. E., Matthews, T., LaMarca, A., Tabert, J., and Powledge, P. 2005. Location disclosure to social relations: why, when, & what people want to share. CHI '05. P. Kumaraguru, J. Cranshaw, A. Acquisti, L. Cranor, J. Hong, M. A. Blair and T. Pham. School of Phish: A Real-World Evaluation of Anti-Phishing Training. SOUPS 2009. Optional reading: HCI methods and experimental design; Research Methods, Chapter 9 Ethnography; Research Methods, Chapter 11 Analyzing Qualitative Data
Thursday, September 29	Warning design study, Part 1	Required reading: S. L. Garfinkel and R. C. Miller. Johnny 2: A User Test of Key Continuity Management with S/MIME and Outlook Express. SOUPS 2005. The Johnny 2 Construction Kit for Testing Email Security from the SOUPS 2006 Security User Studies Workshop User Studies Construction Kits collection S. Schechter, R. Dhamija, A. Ozment, and I. Fischer. The Emperor's New Security Indicators: An evaluation of website authentication and the effect of role playing on usability studies. 2007 IEEE Symposium on Security and Privacy, May 20-27, 2007, Oakland California. - Also read and comment on Andrew Patrick's Commentary on Research on New Security Indicators Research Methods, Chapter 10 Usability Testing Optional reading: Warnings Homework 4 due - Reading summaries from 9/27, 9/29 [+ summary of 1 optional reading]
Tuesday, October 4	Security warnings Warning design study: part 2 Guest lecture: Cristian Bravo-Lillo	Required reading: S. Egelman, L. Cranor, and J. Hong. You've Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. CHI 2008. J. Sunshine, S. Egelman, H. Almuhimedi, N. Atri, and L. Cranor. Crying Wolf: An Empirical Study of SSL Warning Effectiveness. USENIX Security 2009. C. Bravo-Lilli, L. Cranor, J. Downs, and S. Komanduri. Bridging the gap in computer security warnings: A mental model approach. IEEE Security & Privacy, March-April 2011. Optional reading: HCI methods and experimental design
Thursday, October 6	Privacy and mobile and ubiquitous computing [student presentations: Wiese/Cranshaw]	Required reading: G. Iachello, I. Smith, S. Consolvo, M. Chen, and G. Abowd. Developing Privacy Guidelines for Social Location Disclosure Applications and Services. In Proceedings of the Symposium On Usable Privacy and Security 2005, Pittsburgh, PA, July 6-8, 2005. M. Benisch, P.G. Kelley, N. Sadeh, and L.F. Cranor. Capturing location-privacy preferences: quantifying accuracy and user-burden tradeoffs. Personal and Ubiquitous Computing. Published online 07 December 2010. Optional reading: Privacy in mobile and ubiquitous computing Homework 5 due - Reading summaries from 10/4, 10/6; observations report (counts as extra homework) [+ summary of 1 optional reading]
Tuesday, October 11	Privacy policies [student presentations: Samat/Gordon]	Required reading: Security and Usability, Chapter 22 Privacy Policies and Privacy Preferences (L. Cranor) Security and Usability, Chapter 33 Usability and Privacy: A Study of Kazaa P2P File Sharing (N. Good and A. Krekelberg) Optional reading: Privacy policies One-page project proposal due
Thursday, October 13	Privacy software [student presentations: Hiruncharoenvate/Leon]	Required reading: Security and Usability, Chapter 20 A User-Centric Privacy Space Framework (B. Brunk) Security and Usability, Chapter 23 Privacy Analysis for the Casual User Through Bugnosis (D. Martin) Security and Usability, Chapter 26 Anonymity Loves Company: Usability and the Network Effect (R. Dingledine and N. Mathewson) Optional reading: Privacy Homework 6 due - Reading summaries from 10/11, 10/13 [+ summary of 1 optional reading]
Tuesday, October 18	Web browser privacy and security [student presentations: Balebako] What is a browser? Guest lecture: Aleecia McDonald	Required reading: Security and Usability, Chapter 25 Social Approaches to End-User Security and Privacy Management (J. Goecks and E. Mynatt) Security and Usability, Chapter 28 Firefox and the Worry-free Web Z. Ye, S. Smith, and N. Anthony. Trusted Paths for Browsers. ACM Transactions on Information System Security. 8 (2): 153-186. May 2005. Optional reading: Web browser privacy and security
Thursday, October 20	PKIs and secure communication [student presentations: Maass/Ur]	Required reading: Security and Usability, Chapter 16 Making the Impossible Easy: Usable PKI (D. Balfanz, G. Durfee, and D.K. Smetters) Security and Usability, Chapter 30 Embedding Security in Collaborative Applications: A Lotus/Domino Perspective (M.E. Zurko) A. Studer, C. Johns, J. Kase, K. O'Meara, L. Cranor. A Survey to Guide Group Key Protocol Development. Annual Computer Security Applications Conference (ACSAC) 2008. Optional reading: PKIs and secure communication Homework 7 due - Reading summaries from 10/18, 10/20 [+ summary of 1 optional reading]
Tuesday, October 25	Trust and semantic attacks [student presentations: Das/McConahy]	Required reading: Security and Usability, Chapter 5 Designing Secure Systems that People will Trust (A. Patrick, P. Briggs, and S. Marsh) Security and Usability, Chapter 14 Fighting Phishing at the User Interface (R. Miller and M. Wu) Security and Usability, Chapter 29 Usability and Security at Microsoft (C. Nodder) Optional reading: Trust and semantic attacks
Thursday, October 27	User education [student presentations: Baik/Zeng]	Required reading: Teaching Johnny Not to Fall for Phish C. Herley, So Long, and No Thanks for the Externalities: the Rational Rejection of Security Advice by Users. NSPW 2009. Optional reading: User education Homework 8 due - Reading summaries from 10/25, 10/27 [+ summary of 1 optional reading]
Tuesday, November 1	Progress report presentations	Progress reports due
Thursday, November 3	Progress report presentations	Required reading: S. Schechter Common Pitfalls in Writing about Security and Privacy Human Subjects Experiments, and How to Avoid Them, 2009. Security and Usability, Chapter 6 Evaluating Authentication Mechanisms (K. Renaud)
Tuesday, November 8 [election day]	Passwords Guest lecture: Rich Shay	Required reading: Security and Usability, Chapter 7 The Memorability and Security of Passwords (J. Yan, A. Blackwell, R. Anderson, and A. Grant) Security and Usability, Chapter 32 Users are not the Enemy (A. Adams and M.A. Sasse) Optional reading: Authentication
Thursday, November 10	Graphical passwords [student presentations: Xu/Yoon]	Required reading: Security and Usability, Chapter 9 Graphical Password Schemes (F. Monrose and M. K. Reiter) Chiasson, S., Biddle, R., and van Oorschot, P. C. 2007. A second look at the usability of click-based graphical passwords. In Proceedings of the 3rd Symposium on Usable Privacy and Security (Pittsburgh, Pennsylvania, July 18 - 20, 2007). SOUPS '07. E. Hayashi, J. Hong, N. Christin. Security through a different kind of obscurity: Evaluating Distortion in Graphical Authentication Schemes. CHI 2011. Optional reading: Authentication Homework 9 due - Reading summaries from 11/3, 11/8, 11/10 [+ summary of 1 optional reading]
Tuesday, November 15	Challenge questions and password alternatives [student presentations: Graves/Xia]	Required reading: Security and Usability, Chapter 8 Designing Authentication Systems with Challenge Questions (M. Just) S. Schechter, A.J. Brush, and S. Egelman. It's no secret Measuring the security and reliability of authentication via 'secret' questions. IEEE Symposium on Security and Privacy 2009. S. Panjwani and E. Cutrell. Usably Secure, Low-Cost Authentication for Mobile Banking. SOUPS 2010. Optional reading: Authentication
Thursday, November 17	Access control and policy configuration [student presentations: Sleeper/Wang]	Required reading: R. Reeder, L. Bauer, L. Cranor, M. Reiter, K. Vaniea. More than skin deep: Measuring effects of the underlying model on access-control system usability. CHI 2011. L. Bauer, L. Cranor, M. Reiter, and K. Vaniea. Lessons learned from the deployment of a smartphone-based access-control system. SOUPS 2007. L. Bauer, L. Cranor, R. W. Reeder, M. Reiter, and K. Vaniea. A user study of policy creation in a flexible access-control system. CHI 2008. Optional reading: Access control and policy management Homework 10 due - Reading summaries from 11/15, 11/17 [+ summary of 1 optional reading]
Tuesday, November 22	Tools for security administration [student presentations: Hibshi/Vidas]	Required reading: Security and Usability, Chapter 18 Security Administration Tools and Practices (E. Kandogan and E. Haber) G. Conti, M. Ahamad, and J. Stasko. Attacking Information Visualization System Usability Overloading and Deceiving the Human. In Proceedings of the Symposium On Usable Privacy and Security 2005, Pittsburgh, PA, July 6-8, 2005. Optional reading: Tools for security administration
Thursday, November 24	Thanksgiving break, no class
Tuesday, November 29	Biometrics [student presentations: Owusu/Lee]	Required reading: Security and Usability, Chapter 10 Biometric Authentication (L. Coventry) Security and Usability, Chapter 11 Identifying Users from Their Typing Patterns (A. Peacock, X. Ke, and M. Wilkerson) Optional reading: Authentication Homework 11 due - Reading summaries from 11/22, 11/29 [+ summary of 1 optional reading]
Thursday, December 1	no class
Tuesday, December 6	TBA
Thursday, December 8	TBA

This class will have no final exam, however, the final exam period December 13, 1-4 pm in TBA, will be used for final project presentations. Final project papers will be due at the exam period.

Course Requirements and Grading

You are responsible for being familiar with the university standard for academic honesty and plagiarism. Please see the CMU Student Handbook for information. In order to deter and detect plagiarism, online tools and other resources may be used in this class. Students caught cheating or plagiarizing will receive no credit for the assignment on which the cheating occurred. Additional actions -- including assigning the student a failing grade in the class or referring the case for disciplinary action -- may be taken at the discretion of the instructor.

For students taking the 12-unit version of this course, your final grade in this course will be based on:

40% Project
35% Homework
15% Presentation
10% Class participation

For students taking the 9-unit version of this course, your final grade in this course will be based on:

40% Project
50% Homework
10% Class participation

Homework

Homework assignments for this class will include reading summaries as well as written assignments. All homework is due in printed form in class at 3:00 pm each Thursday (unless otherwise specified). Homework submitted after 3:15 pm will be considered late. Homework will be graded as check-plus (100%), check (80%), check-minus (60%) or 0. If you turn in a complete assignment but provide no interesting insights you will get a check. To earn a check-plus requires that you complete the assignment and provide insightful comments. Late homework will receive one grade lower than it would have otherwise received if it is submitted no later than at the beginning of the next class meeting (after that it will not be accepted). Your two lowest homework grades will be dropped from your homework average.

Students are expected to do reading assignments prior to class so that they can participate fully in class discussions. Students must submit a short summary (3-8 sentences) and a "highlight" for each chapter or article in the reading assignment. The highlight may be something you found particularly interesting or noteworthy, a question you would like to discuss in class, a point you disagree with, etc.

Students taking the 12-unit version of this course are expected to include a summary and highlight for one optional reading of their choice each week. Suggested optional readings are provided, but students may choose other relevant optional readings. All other students are encouraged to review some of the optional readings that they find interesting, but they need not submit summaries or highlights of the optional readings.

Presentation

Each student taking the 12-unit version of this course will be assigned a class lecture to prepare and present (either individually or with a small group of other students). The lecture should be based on the topics covered in that week's reading assignment, but it should go beyond the materials in the required reading. Do not present a lecture that simply summarizes the assigned reading. For example, you might read and present some of the related work mentioned in the reading or that you find on your own (the HCISec Bibliography is a good starting point for finding papers), you might present some of the relevant optional reading materials (feel free to use relevant materials from other weeks), you might demonstrate software mentioned in the reading, you might critique a design discussed in the reading, or you might design a class exercise for your classmates. If the material you present describes a user study, include a detailed description and critique of the study design. As part of your lecture you should prepare several discussion questions and lead a class discussion. You should also introduce your fellow students to terminology and concepts they might not be familiar with that are necessary to understand the material you are presenting. You should email to the instructor a set of PowerPoint slides including lecture notes and discussion questions. These slides will be posted on the class web site. In addition, the instructor may include all or part of your presentation slides and notes in an instructor's guide for future usable privacy and security courses.

Project

Students will work on semester projects in small groups that include students with a variety of areas of expertise. A choice of projects will be provided and students will be given an opportunity to indicate their preferences before projects are assigned. Students who have their own ideas for projects should discuss them with the instructor early in the semester. As part of the project students will:

Return their project preference form by September 27 so that they can be assigned to a project by September 29.
Submit a one-page project proposal by October 11. The proposal should describe the system you propose to design or evaluate, discuss what you hope to learn from your user study and/or the hypotheses you plan to test, and provide and overview of your preliminary user study plan (what types of tasks will you have participants do? what types of people will you recruit? will you use a finished software product, prototype, paper prototype, etc. in your user study? will this be a between-subjects or within-subjects study? will you conduct your study online, in a lab, with paper surveys?)
Complete an IRB application with all necessary attachments and submit it to IRB as early in the semester as possible.
Design all questionnaires, scripts, scenarios, interview protocols, etc. necessary to carry out the user study.
Develop any prototypes necessary to carry out the user study.
Pilot test the user study protocol on at least two people (can be members of the class from other project groups) and refine it based on these tests (or other similar piloting as appropriate if this is not a lab study).
Give a 10-15 minute progress report presentation on November 1 or 3.
Submit a written progress report by November 1. Your written progress report and presentation should describe your progress to date and any problems you have run into that you would like some advice on. In addition, the written report should include a revised user study plan and the details of your initial pilot user study, including the study design and scripts (and results if you have already completed the initial study)
Conduct a study using the revised protocol with at least 6 subjects (or more if this is not a lab study). (Optionally, you can conduct a larger study that would be likely to lead to publishable results. If your study has only 6 subjects, most likely this will be useful mostly as a pilot study, and should be positioned as such in your paper.)
Give a 15-minute final project presentation during the final exam period.
Write a paper giving an overview of the proposed study, what you hope to learn from it, what you learned from the pilot study, etc. and submit it by 1 pm on December 13 in both electronic and printed form. Your IRB forms, survey forms, etc. should be included as appendices.

Students are encouraged to submit their project to the Symposium On Usable Privacy and Security as either a paper or poster. A paper submission will likely require some additional work after the end of the semester. To submit a poster will require only submitting a 2-page abstract. The instructor will provide funds for one student from each project team to attend the SOUPS conference if their paper or poster is accepted.

Students signed up for the 12-unit version of this course are expected to play a leadership role in a project group and write a project paper suitable for publication. Unless your group has only students signed up for the 9-unit course in it, that means your final paper should be written in a style suitable for publication at a conference or workshop. The conference papers in the optional readings provide some good examples of what a conference paper looks like and the style in which they are written. In addition to describing what you did in your study, your paper should include a related work section and properly-formatted references. Papers should follow the SOUPS 2011 technical papers formatting instructions. If you have identified an alternative relevant conference and would prefer to use that conference's submission format for your paper, please discuss it with the instructor.