Job Talk 2012

This is a variant of the job talk I gave at the various companies and universities I interviewed with during my 2012 academic job search.

Transcript

1. Patrick Gage Kelley @patrickgage DESIGNING PRIVACY INTERFACES Supporting user understanding and control
2. Privacy and security HCI, specifically usability Design / new media arts 2
3. Passwords Encountering Stronger Password Requirements: User Attitudes and Behaviors. Shay et al. SOUPS 2010. Of Passwords and People: Measuring the Effect of Password-Composition Policies. Komanduri et al. CHI 2011. Guess again (and again and again): Measuring password strength by simulating password- cracking algorithms. Kelley et al. OAKLAND 2012. CHI 2011 HONORABLE MENTION 3
4. Location sharing Empirical Models of Privacy in Location Sharing. Toch et al. UBICOMP 2010. Location-Sharing Technologies: Privacy Risks and Controls. Tsai et al. I/ S 2010. Who’s Viewed You? The Impact of Feedback in a Mobile-location System. Tsai et al. CHI 2009. Capturing Social Networking Privacy Preferences... Ravichandran, et al. PETS 2009. 4
5. Location sharing with advertisers When Are Users Comfortable Sharing Locations with Advertisers? Patrick Gage Kelley, Michael Benisch, Lorrie Faith Cranor, and Norman Sadeh. CHI 2011. 5
6. 6 Anti-phishing education
7. Social network friend grouping An Investigation into Facebook Friend Grouping Patrick Gage Kelley, Robin Brewer, Yael Mayer, Lorrie Faith Cranor, and Norman Sadeh. INTERACT 2011. INTERACT HONORABLE MENTION Paul Adams. The Real Life Social Network 7
8. Twitter RT @IWantPrivacy: Widespread Violation of Privacy Settings in the Twitter Social Network. Brendan Meeder, Jenn Tam, Patrick Gage Kelley, and Lorrie Faith Cranor. W2SP 2010. 8
9. Journalism 9
10. New media arts 10
11. Today I want to focus on two projects Online privacy policies Android permissions 11
12. Design of a “nutrition label” for privacy 12
14. Federal Trade Commission Privacy Online: A Report to Congress June 1998 14 In light of the Commission's findings and significant consumer concerns regarding privacy online, it is evident that substantially greater incentives are needed to spur self- regulation and ensure widespread implementation of basic privacy principles. “ ”
15. 15 FTC. Privacy Online: A Report to￿Congress. June 1998. EPIC. Surfer Beware III: Privacy Policies without Privacy Protection. 1999 C. Jensen, C. Potts. Privacy Policies as Decision-Making Tools: An Evaluation of Online Privacy Notices. CHI 2004 A. McDonald, L. Cranor. The Cost of Reading Privacy￿Policies. I/S. 2008. upward of 85% – collect personal information from consumers. only 14% – provide any notice with respect to their information practices ~2% – provide notice by means of a comprehensive privacy policy. However, by 1999 privacy policies were found on over 80% of top websites. The average Flesch-Kincaid score required for the top 50 internet privacy policies (2003) was 34.2 Time = 244/hours year (national opportunity cost for time to read policies: $781 billion)
16. is is what consumers are up against. 16
17. Can we build a better privacy policy? 17
18. Can more intentionally designed, standardized privacy policy formats benefit consumers? 18
19. Can more intentionally designed, standardized privacy policy formats benefit consumers? • Ease of understanding • Speed of information- nding • Ability to make comparisons • Consumer opinion 19
20. Challenges 20 • People are not familiar with privacy terminology • Context matters • Not enough to know only type of data collected and how data is used • Need to know which data are used for what purposes as companies use some data for some purposes and other data for other purposes • Privacy policies are complex • People don’t understand privacy implications
21. Platform for Privacy Preferences P3P 21 A machine-readable privacy language <purpose> admin, current, develop,... <recipient> ours, other, public,... <data> physical, cookies, computer,...
23. Standardized format • People learn where to look • Side-by-side comparisons Standardized language • People learn the terminology Brief • People can get their questions answered quickly 23 Towards a privacy “nutrition label”
24. 24 Evolution of a Prototype Financial Privacy Notice February 2006 Kleimann communication group
25. 25 KCG. Evolution of a Prototype Financial Privacy Notice. 2006 Instructions Possible types of information they collect Purpose of the policy Will they share this information “for this purpose” Can you opt-out? Contact information
26. 26 Iterative design approach 5 focus groups • 7-11 participants each • explored attitudes towards privacy policies • tested understanding of labels and symbols Patrick Gage Kelley, Joanna Bresee, Lorrie Faith Cranor, and Robert W. Reeder. A "Nutrition Label" for Privacy. SOUPS 2009.
28. What we collect How we use your information Who shares your information Provide service and maintain site Research and development Marketing Telemarketing Profiling not linked to you Profiling linked to you Other companies Public forums Contact information Content Cookies Demographic information Social security no. and gov't ID Preferences Purchase and financial data Web browsing information Unique identifiers Understanding this privacy report Data is collected and used in this way. Your data will not be used in this way unless you opt-in. You can opt-out of this data use. You can opt-in or opt-out of some uses of this data.
29. The Acme Policy types of information contact information cookies demographic information financial information health information preferences purchasing information social security number & govt ID your activity on this site your location how we use your information provide service & maintain site research & development marketing telemarketing profiling who we share your information with other companies public forums
30. Design Evolution 30 Final Proposed Design Design Evolution Acme Privacy Policy
31. 31 Laboratory Study • 24 participants • within subjects design to compare label and text policies • 8 tasks, measured time and accuracy • 6 opinion questions Iterative design approach 5 focus groups • 7-11 participants each • explored attitudes towards privacy policies • tested understanding of labels and symbols Patrick Gage Kelley, Joanna Bresee, Lorrie Faith Cranor, and Robert W. Reeder. A "Nutrition Label" for Privacy. SOUPS 2009.
32. Comparison information finding Questions answered correctly Privacy Label Full text Policy
33. Comparison information finding Questions answered correctly Privacy Label Full text Policy p = 0.021 p = 0.0036
34. Standardized label 34
35. 35 Removes wiggle room and complicated terminology by using four standard symbols
36. 36 Allows for quick high-level visual feedback by looking at the overall intensity of the page
37. 37 Allows for information to be found in the same place every time
38. 38 Can be printed, fits in a standard browser window
39. 39 A legend explains each of the four symbols, a definition clearly explains each term
40. 40 Amazon’s Mechanical Turk • 764 participants • Between subjects design • Measured time, accuracy, and enjoyability on information finding and comparison tasks • Average time to complete ~15 minutes User testing Standardizing Privacy Notices: An Online Study of the Nutrition Label Approach. Patrick Gage Kelley, Lucian Cesca, Joanna Bresee, and Lorrie Faith Cranor. CHI 2010.
41. Study questions 41 Demographics Internet and privacy Simple tasks • Can be answered from single row or column Complex tasks • Interaction between rows and columns Single policy likeability Comparison tasks Policy comparison likeability
42. 42 Five formats compared Standardized label Standardized short label Standardized short text Full policy text Layered text
43. 43 table table text text table (with text) Five formats compared Standardized label Standardized short label Standardized short text Full policy text Layered text
44. 44 table table text text table (with text) standardized standardized standardized real-world real-world Five formats compared Standardized label Standardized short label Standardized short text Full policy text Layered text
45. Standardized label 45
46. Standardized short label 46
47. Standardized labels compared 47
48. 48 Short standardized text
49. Full policy text 49
51. 51 Layered policy
52. 52 Std. Label Std. Short Label Std. Short Text Full Policy Text Layered Text Percentage correct Overall accuracy results ANOVA significant at p < 0.05, F(4; 1094) = 73.75 std label vs. full text, p < 0.05, t(510) = 14:4, std. short label vs. full text p < 0.05, t(490) = 12.9, std. short text vs. full text p < 0.05, t(491) = 14.3 layered vs. full text policy p = 0.83, t(314) = -0.21
53. 53 Std. Label Std. Short Label Std. Short Text Full Policy Text Layered Text Timing results ANOVA on the log-normalized time information p < 0.0001 Standardized all took less time sig. p < 0.05, layered at p = 0.025 std. label, t(348) = 5.36, std. short label t(327) = 6.01, std. short text t(329) = 4.55, layered t(238) = 2.25
54. 54 Enjoyability – single policy ANOVA, F(4; 756) = 4.25; p < 0.05
55. 55 Enjoyability – comparisons ANOVA, F(4; 756) = 10.65; p < 0.05
56. Standardized formats outperformed text and layered formats Structured information presentation Clear labeling of information that is not used or collected Standardized terminology to minimize length and increase the clarity of the text Definitions of standardized terms 56
57. Minor differences between standardized formats Standardized table presents holistic view of policy Short table takes up less space but sometimes makes comparison tasks and tasks about data not collected more difficult Text doesn’t scale well for complex policies, people more likely to miss text in the middle of paragraphs 57
59. Layered policy did not perform well Layered performed similarly to full policy Some information was not in layered policy yet few people clicked through to full policy to look for it Layered not standardized enough – many differences between companies 59
62. Participant comments The full policy text described as: torture to read and understand likened them to Japanese Stereo Instructions The standardized-format were more complimentary: This layout for privacy policies is MUCH more consumer friendly. I hope this becomes the industry standard 62 “ “ ” ” “ ”
63. Can more intentionally designed, standardized privacy policy formats benefit consumers? Yes. • Ease of understanding • Speed of information- nding • Ability to make comparisons • Consumer opinion 63
64. 64 1st Place, ACM SIGCHI Student Research Competition. 2009. 1st Place, ACM Grand Finals Student Research Competition. 2010. Privacy Papers for Policy Makers Honorable Mention, NYU-poly Computer Security Awareness Week
65. Please keep in mind that any opt-out choices you make will not apply in situations where (a) you either have made, simultaneously make, or later make a specific request for information from a member of The Bell Group, (b) The Bell Group uses your personal information for either “Operational Uses” or “Fulfillment Uses” (as described above in A3), (c) you either have engaged, simultaneously engage, or later engage in either Non-Registered Transactions or Sponsored Activities (as described above in A3), or (d) The Bell Group shares your personal information under the provisions of A3 above with respect to “Companies That Facilitate Communications and Transactions With You,” “Companies That You Previously Authorized to Obtain Your Information,” “Purchase or Sale of Businesses,” or “Disclosures to Comply with Laws and Disclosures to Help Protect the Security and Safety of Our Web Sites, The Bell Group and Others.” Also, any opt-out choices you make will not apply to personal information that you provide about other persons, but these other persons will have the
66. Online privacy policies Android permissions
67. Android application permissions 67 Patrick Gage Kelley, Lorrie Faith Cranor, Sunny Consolvo, Jaeyeon Jung, Norman Sadeh, David Wetherall. A conundrum of permissions... FC USEC 12
68. 250,000 applications 6 billion downloads ...and no application review 68
69. 250,000 applications 6 billion downloads ...and no application review 69 and Bouncer
70. The market requires users to make two choices when reviewing potential applications for their device. 1. Do I believe this application will compromise the security and function of my phone if I install it? 2.Do I trust this developer and their partners with access to my personal information? 70
71. Android permissions screens 71
72. Permissions interface issues 72 - Information is hidden away - No clear way to cancel - Unclear terms and concepts - Unclear what app doesn’t do - No sense of importance, necessity, purpose - No way to opt-out
73. 73 Android permissions screens
74. 74 Android permissions screens
75. User interviews 75 Interviewed 20 Android smartphone users from Pittsburgh and Seattle Semi-structured interview methodology focused on ecosystem wide issues: - What do they think of Android generally? - Why and how do they select apps to install/purchase? - Do they read and understand permissions screens? - Are they concerned about malicious applications? - Are tools/info needed to help with app privacy/security?
76. Why and how do they select apps to install/ purchase? - The reviews and star-ratings, word of mouth from friends, and those who don’t see anything sketchy on the permissions list - Nearly all participants don’t buy apps, so since it is free, they try it, and later delete it Do they read and understand permissions screens? - Many said they try, most don’t believe they understand the terms used, and haven’t tried to learn them - They trust the reviews more - They don’t understand why the apps need such access 76
77. Are they concerned about malicious applications? - Largely unconcerned, believe Android is protecting them with app review for usability, bugs, viruses - Are concerned in general about technology, most refused to do banking on their phones Are tools/info needed to help with app privacy/security? - Most said they would be interested in better app reviews, or an app that checks their phone, a few had tried similar tools, installed anti-virus software 77
78. Network communication: full Internet access 78 That you can have access to all kinds of websites, even the protected ones.” –P1 I would say, this just requires a data plan, and you would need to have Internet access.” –P6 Any app that needs to get information from somewhere other than that is local on the phone.” –P7 “ “ “
79. Phone calls: read phone state and identity 79 I would assume it would probably be along the lines of, it knows when my phone is sleeping or in use or in a phone call, and the type of phone” –P2 So it knows whether or not I am in the middle of a call? I don’t really know what that part [identity] means.” –P13 If you are on the phone maybe it shuts itself off... Maybe like your carrier? Hopefully not like who you are.” –P19 “ “ “
80. Your accounts: act as an account authenticator 80 That I don’t like, I don’t know what it means, ... my impression is that instead of me being able to authorize something, that application is saying it can.” –P3 That freaks me out. What does that mean exactly, cause I am not quite sure.” –P12 I don’t know, I guess it is in charge of whatever accounts you open up.” –P18 “ “ “
81. Overall, users are not currently well prepared to make informed privacy and security decisions around installing applications from the Android market. 81
82. Can we create a better designed permissions display for mobile apps? ...ongoing 82
83. Online privacy policies Android permissions
84. What’s next?
85. What’s next? General design principles Additional domains User Controllable Preference Learning
86. General Design Principles standardization extended explanation decision removal automation interface nudging holistic views simplified design
87. Additional Domains Password management Legal documents Structured text: news Social networks Friend grouping Twitter regrets Privacy settings
88. User Controllable Policy Learning leveraging algorithm automation in a way users can understand
89. http://cups.cs.cmu.edu Thesis Committee Lorrie Faith Cranor Norman Sadeh Alessandro Acquisti Sunny Consolvo Patrick Gage Kelley @patrickgage me@patrickgage.com patrickgagekelley.com Privacy nutrition labels Joanna Bresee, Aleecia McDonald, Rob Reeder, Sungjoon Steve Won Android app permissions Jaeyeon Jung, David Wetherall, Tim Vidas Location sharing Michael Benisch, Janice Tsai, Eran Toch, Paul Hankes Drielsma, Jialiu Lin, Jason Hong Passwords Michelle Mazurek, Saranga Komanduri, Rich Shay, Blase Ur, Lujo Bauer Twitter/Facebook Manya Sleeper, Justin Cranshaw, Yang Wang, Yael Mayer, Robin Brewer New Media Arts Golan Levin, Danny Rashid, Matthew Kay, Polo Chau, Sue Ann Hong