Ensuring Value in our Work: The ROI of Design (FEDC 2018)

This was a talk at FEDC 2018 about how to measure the ROI of design.

1. ENSURING VALUE IN OUR WORK The ROI of Design JARED PONCHOT // @JPONCH // FEDC 2018
2. ‣Understanding the ROI of an improved User Experience WHAT’S MOST IMPORTANT FOR YOUR AGENCY RIGHT NOW?
3. THE ROI OF DESIGN?
4. ROI=
 RETURN ON INVESTMENT
5. the resources (e.g. time, money) we invest in design will wind up getting us even more money in return? HOW CAN WE KNOW THAT …
7. “ — ALBERT EINSTEIN If you can’t explain it to a six year old, you don’t understand it yourself.
8. A TAXONOMY FOR DESIGN VALUE
9. ‣ Brand ‣ Innovation ‣ Efficiency ‣ Satisfaction ‣ Sustainability
10. BRAND
11. “ — HARTMUT ESSLINGER, FROG DESIGN Money buys, but emotion sells.
13. Desirability Studies
14. INNOVATION
15. ‣ Making strategic initiatives possible ‣ Creating new products, services, etc. ‣ Enabling entry into new markets INNOVATION THROUGH
16. Re-imagining problems
17. Tesco Subway Store in South Korea image from littledoremi.com
18. from evigo.com
19. from jcdecaux-oneworld.com
20. Intellectual Property
21. EFFICIENCY
22. 25% FAIL OUTRIGHT IT PROJECTS PROVIDE NO ROI (IF THEY SUCCEED) 20-25% REQUIRE SIGNIFICANT REWORK 50% Based on a 2009 study
23. Poor requirements
24. Editorial & process improvements
25. Customer service requests
26. SATISFACTION
27. “ — JEROME KATHMAN, CEO OF LPK Consumer-preferred design outperforms management-preferred design.
28. Usability
29. SUSTAINABILITY
30. YOUR USERS
 desirability YOUR BUSINESS
 viability YOUR BUILD TEAM
 feasibility
31. ‣ Is it changing how customers see our brand? ‣ Is it helping us innovate? ‣ Is it increasing efficiency? ‣ Is it improving satisfaction? ‣ Is it making things more sustainable?
32. MEASURING IS HARD
33. Projects often begin with a measurement to improve
34. Simpson’s Paradox
35. 7.9% 11%
36. 22.9% 2.8% 0% 11.3%
39. “If an unfriendly foreign power had attempted to impose these bad schools on us, we might well have viewed it as an act of war.”
40. MATH SCORES 40pts VERBAL SCORES 50pts
41. Bad Teachers
42. More test takers
43. LOW INCOME INCOME: LURKING VARIABLE MIDDLE INCOME HIGH INCOME
44. LOW INCOME INCOME: LURKING VARIABLE MIDDLE INCOME HIGH INCOME
45. LOW INCOME INCOME: LURKING VARIABLE MIDDLE INCOME HIGH INCOME
46. LOW INCOME INCOME: LURKING VARIABLE MIDDLE INCOME HIGH INCOME
47. 448/8,000 VISITS PAGE A 5.6% 584/8,000 VISITS 7.3% PAGE B
48. PAGE A PAGE B
49. PAGE A PAGE B FACEBOOK 300/6,000 5%
50. PAGE A FACEBOOK 30/750 4% PAGE B FACEBOOK 300/6,000 5%
51. PAGE A GOOGLE ADS 60/1,000 6% FACEBOOK 30/750 4% PAGE B FACEBOOK 300/6,000 5%
52. PAGE A GOOGLE ADS 60/1,000 6% GOOGLE ADS 40/800 5% FACEBOOK 30/750 4% PAGE B FACEBOOK 300/6,000 5%
53. PAGE A GOOGLE ADS 60/1,000 6% EMAIL 90/1,000 9% GOOGLE ADS 40/800 5% FACEBOOK 30/750 4% PAGE B FACEBOOK 300/6,000 5%
54. PAGE A GOOGLE ADS 60/1,000 6% EMAIL 90/1,000 9% GOOGLE ADS 40/800 5% EMAIL 516/6,450 8% FACEBOOK 30/750 4% PAGE B FACEBOOK 300/6,000 5%
56. DATA != MEANING
57. Confirmation Bias
58. “ — UPTON SINCLAIR It is difficult to get a man to understand something, when his salary depends on his NOT understanding it.
59. Gestalt Principle: Isomorphism
61. Photos from Reddit
62. “ — JACK HANDY To me, clowns aren't funny. In fact, they're kind of scary. I've wondered where this started and I think it goes back to the time I went to the circus, and a clown killed my dad.
63. Photo by Ghida Basma
64. Availability Heuristic
66. “ — ERIKA HALL We tend to measure what we can easily count, but not necessarily what counts.
67. Goodhart’s Law
68. MORE DATA != BETTER DECISIONS
69. Photos by Reuters / Bassim Shati and from https://www.huffingtonpost.co.uk/2014/05/07/working-at-amazon-like_n_5278503.html
70. The allure of cheap science
71. SCIENCE & ART
73. Sunset photo from http://www.city-data.com/picfilesv/picv16390.php
75. From https://reasonandreflection.wordpress.com/tag/atomic-bomb/
76. 1. Mystery 2. Hypothesis 3. Repeated Experimentation 4. Theory 5. Fact SCIENTIFIC METHOD
77. 1. Mystery
78. 2. Hypothesis
79. 3. Repeated Experimentation
80. 4. Theory
81. 5. Fact
82. Repeat
83. EPISTEMOLOGY
84. How do we know things?
85. From wallpapercave.com
86. “ — JEFF BEZOS “Most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you’re probably being slow.
87. Trust
88. TAKE RISKS
89. The value of design
90. “ — ANNE LAMOTT Perfectionism is the voice of the oppressor, the enemy of the people. It will keep you cramped and insane your whole life, and it is the main obstacle between you and a shitty first draft.
91. THANKS! Jared Ponchot @jponch