Best Practices from 10 Years of UX Research

For UXPin on Sep 14th, 2016

1. Best Practices From 10 Years of Remote UX Research
2. thx
3. UX Research vs. Remote UX Research vs. User Research
4. Watching people use interfaces in order to design better interfaces
5. 60 YEARS AGO
6. LET’S RECORD HUMANS USING INTERFACES 30 YEARS AGO
7. Vs. 10 YEARS AGO
8. 1% 
 Humans observing humans 99% 
 Data
9. • UX Research (Facebook, Google) • Experience Research (AirBnB) • Insights (Netflix) • Design Research (Uber) • User Research (So Many People) INTERCHANGEABLE TERMS
11. Analytics Data Science Quantitative Research User Research UX Research Usability User Research 1:1 Observation Lean Research
12. TEXT Geography of remote UX Research
14. Btw I’m Nate Bolt
17. PLEASE BUY THIS BOOK
19. ethn.io
20. How To Plan, Recruit, and Conduct Remote UX Research
22. 1. Ask the right questions 
 AKA Conversations with Stakeholders
23. Figuring out the right question(s) is tricky
24. Quantitative research lets us test our assumptions, and qualitative research lets us find out what we don't know. “ ”
25. What do our customers/ users/humans want? “ ”
26. • Remoteresear.ch site • Mental Models by Indi Young • Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's Guide to User Research A Deeper Read
27. 2. Try to Incorporate Data •Surveys •Experiments (A/B, etc) •System data •Purchase Behavior
28. 3. Identify Testing Goals
29. What do our customers/ users/humans want? “ ”
30. (and write them down in a G&O and then in a Moderator Script)
31. remoteresear.ch/category/samples
32. 3. Recruit real humans ethn.io
33. The Recruiting Spectrum Grab Someone in Your Office Live Customer Intercept Recruiting Agency
34. 4. Combine Tools & Methods
35. Observe Behavior
36. 5. Collaborate don’t report
37. And how does this fit into the design process?
38. Understand Prototype Build
39. Understand • review existing research • focus groups • interviews • ethnography • surveys • card sorts • data analysis • diary studies
40. Prototype • participatory design • collaborative design • brainstorming techniques • heuristic evaluation • paper prototype testing • static mock testing
41. Build • usability testing • heuristic evaluation • diary study • A/B test • field testing
42. 1. Identify Testing Goals 2. Incorporate Data 3. Recruit Real Humans 4. Combine tools & methods 5. Collaborate Don’t Report How To Plan, Recruit, and Conduct Remote UX Research
43. Pitfalls And Now Some Common
44. Decision Theory & Adaptive Systems Group
46. • Hypothetical Questions • A False Premise • Opinions are worthless Some Common Pitfalls of Remote UX Research
47. • Behavior • Context • Behavior Best Practices From 10 Years of Remote UX Research
48. • Bandwidth (games) • Security (prototypes) • Translation (time) • Rapport (lacking) Even More Pitfalls
49. How & When to Run Rapid Remote UX Research
51. 46 Million Americans That’s 22 Million Households 15% of the country
53. Now what percentage of those 46 million were employed within a year of receiving food assistance?
54. > 80%
55. So these are working people. People with families. Teachers, etc. How do they apply?
56. 90 Minutes / 53 Days up to
57. Alan Williams Design Lead, SNAP Caroline Hays Health Focus Area Intern Jacob Soloman Policy analyst and project manager
58. SNAP (Supplemental Nutrition Assistance Program) Let’s Try the Interface for a Application
62. Just Stop
66. 30 phone interviews 10 minutes each 58 unique experiences in California
67. And 15 Follow-Up Phone Interviews
68. Quant & Qual
73. 11 Minutes / 53 Days (?)
74. • 30 Remote Interviews • Real context • 15 Follow-Ups • Little to No Report • Input straight to designer & engineer Best Practices From 10 Years of Remote UX Research
75. • Easy, cheap, & fast phone interviews • Quick “Guerilla” Human Observation • No statistical significance • Behavior repeats over small samples • Combine with other inputs Best Practices From 10 Years of Remote UX Research
76. Top Tools for Remote UX Research
77. “At least a little behavioral” - Marv Surveys. Web Analytics.
78. Five Categories •Mobile (spans other categories) •Moderated (screen-sharing + audio) •Self-moderated (usertesting.com) •Automated (loop11, usabilla) •Fancy Analytics on Steroids (crazyegg)
79. remoteresear.ch/tools
80. Conceptual For A Long Time Self-Moderated Moderated Web Analytics Automated Static Automated Live
81. Tasks On Concepts For A Long Time People Talk To A Machine Talk To People Fancy-Ass Analytics Task Bar On Images Task Bar On A Live Site
82. I.A. & Navigation For Ad Agencies Ammunition Insight Just Better Analytics Still More Ammunition More Ammunition
83. Conceptual For A Long Time Self-Moderated Moderated Web Analytics Automated Static Automated Live Deep Behavioral Ux Insight
84. Opinions Conceptual For A Long Time Self-Moderated Moderated Web Analytics Automated Static Automated Live
85. Ad Agencies Love It Conceptual For A Long Time Self-Moderated Moderated Web Analytics Automated Static Automated Live
86. USABILLA Clicks & Questions Heatmaps You do your own recruiting $49 - $199
90. ethn.io
96. If NASA can do THIS from 153 million miles away, we can meet our users where they are, even if we’re a couple thousand miles away.
97. Thanks for attending Go run some awesome remote UX research
98. fin.
99. • Remoteresear.ch site • Mental Models by Indi Young • Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's Guide to User Research A Deeper Read
100. nate@ethn.io @ethnio Nate Bolt