Designing Inclusive Products

Most of us start projects with good intentions—we want to make things welcoming, seamless, and maybe even fun to use. But too often, toxic cultures within tech result in products that have all sorts of biases embedded in them: “smart scales” that assume everyone wants to lose weight, form fields that fail for queer people, image-recognition software that doesn’t work for people of color. As tech becomes increasingly central to our users' days—and intertwined with their most intimate lives—we have more responsibility than ever to consider who could be harmed or left out by our decisions.

Transcript

1. DESIGNING INCLUSIVE PRODUCTS Sara Wachter-Boettcher // June 2018
2. flickr.com/photos/hjjanisch/8606288865/ (CC BY-ND 2.0)
3. 8:00 p.m.
5. 9:03 p.m.
6. To recap: • There’s no way to turn it off • This is dangerous for people with eating disorders • This feels shamey • “Average” calorie counts are wildly inaccurate • Not all calories are equal • A cupcake is not a useful metric • Pink cupcakes are not neutral—they have social and culture encoding (feminine, white, middle class) • This perpetuates diet culture
7. It took one hour to document these flaws.
8. It took three hours to shut the feature down.
9. How many hours did it take to build?
10. flickr.com/photos/3059349393/8000156483/ (CC BY 2.0)
11. ‘‘ Silicon Valley is run by people [who] want to be in the tech business, but are in the people business. They are way, way in over their heads. —Zeynep Tufekci
12. ‘‘ Twitter is failing in its responsibility to respect women’s rights online by inadequately investigating and responding to reports of violence and abuse in a transparent manner. —Amnesty International, 
 March 2018
13. ‘‘ We love instant, public, global messaging… But we didn’t fully predict or understand the real-world negative consequences. We acknowledge that now… 
 We aren’t proud of how people have taken advantage of our service, or our inability to address it fast enough. —Jack Dorsey, March 2018
14. ‘‘ We suck at dealing with abuse and trolls on the platform and we’ve sucked at it for years… We lose core user after core user by not addressing simple trolling issues that they face every day. I’m frankly ashamed of how poorly we’ve dealt with this issue during my tenure as CEO. It’s absurd. There’s no excuse for it. —Dick Costolo, February 2015
15. Screensh0t from a knockoff Peppa Pig video (James Bridle)
16. • Cartoons about torture and suicide • Sexualized characters and themes • Violence and weapons • Kids being tied up and hurt • Kids vomiting and writhing in pain
17. Image: Google I/O
18. Image: Google I/O • How will we trust who we’re talking to? • Is it OK for a bot to pretend it’s human? • What does recording the call mean for privacy? • Is it fair to outsource task to bots and expect low-wage workers to deal with it?
19. We have normalized chronic underinvestment in inclusion and harm prevention.
20. WISHFUL THINKING
22. The more you talk, the smarter Tay gets.
24. ‘‘ We stress-tested Tay under a variety of conditions, specifically to make interacting with Tay a positive experience. —Peter Lee, Microsoft Research
25. ‘‘ We stress-tested Tay under a variety of conditions, specifically to make interacting with Tay a positive experience. —Peter Lee, Microsoft Research
26. From Dan Hon
27. From Dan Hon
28. From Eric Meyer
29. From @oliviasolon
30. From @sallyrooney
31. ‘‘ We talked about getting rid of it but it performs kinda great :/ —Tag Savage, lead writer at Tumblr
32. Inclusion may be inconvenient to your business model.
33. BIASED NORMS
35. ‘‘ We are deeply sorry for this unquestionably serious issue. It is an unfortunate side-effect of the underlying neural network caused by the training set bias, not intended behaviour. — Yaroslav Goncharov, FaceApp CEO
36. ‘‘ We are deeply sorry for this unquestionably serious issue. It is an unfortunate side-effect of the underlying neural network caused by the training set bias, not intended behaviour. — Yaroslav Goncharov, FaceApp CEO
37. From Jacky Alciné
38. From Jacky Alciné
39. ‘‘ We’re also working on longer-term fixes around both linguistics (words to be careful about in photos of people)… and image recognition itself (e.g., better recognition of dark-skinned faces). —Yonatan Zunger, former Googler
40. ‘‘ We’re also working on longer-term fixes around both linguistics (words to be careful about in photos of people)… and image recognition itself (e.g., better recognition of dark-skinned faces). —Yonatan Zunger, former Googler
41. ‘‘ When the person in the photo is a white man, the software is right 99 percent of the time. But the darker the skin, the more errors arise — up to nearly 35 percent for images of darker skinned women. —“Facial Recognition Is Accurate, if You’re a White Guy”
 by Steve Lohr, The New York Times
42. ‘‘ When the person in the photo is a white man, the software is right 99 percent of the time. But the darker the skin, the more errors arise — up to nearly 35 percent for images of darker skinned women. —“Facial Recognition Is Accurate, if You’re a White Guy”
 by Steve Lohr, The New York Times
43. Courtesy of Kodak and Hermann Zschiegner
44. These biases get baked into all kinds of products.
45. flickr.com/photos/147489968@N06/29300385324 (www.paintzen.com - CC BY-ND 2.0) COCO: More than 100,000 images from the web, labeled.
46. flickr.com/photos/147489968@N06/29300385324 (www.paintzen.com - CC BY-ND 2.0) AI trained on it associates kitchen implements with women.
47. HOW WE CHANGE THINGS
48. Practice Process Priorities
49. Practice Process Priorities
50. ‘‘ If you’re not asking yourself “how could this be used to hurt someone” in your design/engineering process, you’ve failed. — Zoe Quinn
51. ASSUME
54. • The user had a good year. • The user wants to relive their year. • The user wants to share their year. • The user’s most popular content is positive.
55. • Identity • Location • Emotional state • Physical state • Personal history • Lifestyle • Goals • Pain points
57. ‘‘ My hair type is what’s called ‘4C hair,’ given the level of coiliness. I learned that I needed to add that to my searches in order to find things. It shouldn’t be that way. — Candice Morgan, 
 Head of Diversity & Inclusion, Pinterest
58. Practice Process Priorities
59. If it’s important, it’s planned.
60. If it’s important, it’s resourced.
61. If it’s important, it’s evaluated.
62. Make inclusive design an explicit part of: • Project roles and responsibilities • Roadmapping and project planning • Use cases and scenarios • Content crits and editing cycles • Project postmortems • Employee evaluations
63. Practice Process Priorities
65. ‘‘ We’re an idealistic and optimistic company. For the first decade, we really focused on all the good that connecting people brings… But it’s clear now that we didn’t do enough. We didn’t focus enough on preventing abuse and thinking through how people could use these tools to do harm as well. —Mark Zuckerberg, April 2018
66. ‘‘ As most companies grow, they slow down too much because they’re more afraid of making mistakes than they are of losing opportunities by moving too slowly. We have a saying: “Move fast and break things.” The idea is that if you never break anything, you’re probably not moving fast enough. —Mark Zuckerberg, 2012
67. ‘‘ As most companies grow, they slow down too much because they’re more afraid of making mistakes than they are of losing opportunities by moving too slowly. We have a saying: “Move fast and break things.” The idea is that if you never break anything, you’re probably not moving fast enough. —Mark Zuckerberg, 2012
68. If you never slow down, you probably have no idea what you’re breaking.
69. Image: Pixabay (CC0)
70. Image: Pixabay (CC0)
71. Image: Josh Mattson (CC - BY NC)
75. our choices hold
 power flickr.com/photos/vchili/6028132840
76. how will you 
 use yours? flickr.com/photos/vchili/6028132840
77. @sara_ann_marie rareunion.com sarawb.com thank you