Building Data Driven Organizations

Given at IT Weekend 2014 in Kiev

Transcript

1. @AbeStanway BUILDING A DATA DRIVEN ORGANIZATION
2. 1. why 2. how
3. 1. why 2. how
4. “DATA IS THE NEW GOLD”
5. Predict the future!
6. Retain Customers!
7. Grow the business!
8. Recommend content!
9. Drive Engagement!
10. unclear paths to $$$
11. IN IT, It’s clear.
12. Data are Dollars
13. . IT Working = +$$$ IT Not Working = -$$$ . .
14. How do you know if your IT is working right now?
15. How do you know if you are earning money right now?
16. KPIs. What are they?
17. Etsy: Literally a Money per second Graph
18. Planet Labs: Literally an Images per day graph
19. What are the Kpis for kips?
20. $ per second items bought per second page requests per second database queries per second memcache hits per second fread() per second
21. If you do not have the data about your infrastructure, it is already broken. LEsson:
23. Test driven development -> data driven development
24. Without data, you are flying blind
25. How do you know you’re hitting your goals?
26. How do you know if You’re making the right ones in the first place?
27. How do you know if you’re still in business?
28. How do you even know what planet you live on?
29. Assumptions are death
30. You need data, yo.
31. 1. why 2. how
32. 1. collect 2. analyze 3. ??? 4. Profit!
33. 1. collect 2. analyze 3. ACT 4. Profit!
34. data that cannot be acted upon should not be analyzed.
36. You are running a business, not an art museum
37. You are Trying to Win the market, not a fields medal
38. This can be disappointing
39. Data SCientist?
40. Data Scientist? Realist.
41. Find a way to Align your employees intellectual curiosity With your Real business needs. LEssoN:
42. Train your organization
43. you need a data culture.
44. “It’s not shipped until it’s monitored”
45. “If you are not looking at Dashboards, you are not doing your job”
46. Building instrumentation and watching dashboards are hard And Time consuming
47. App code -> statsD -> Graphite -> Dashboards -> Insights by hand by hand by hand by hand by hand
48. Developers just want to code
49. Let’s automate
50. Which is easier to automate? Insights or data collection?
51. Insights are sexy and fun
52. Collection is hard And unsexy
53. Collection is hard And Boring
54. Collection is hard And unsexy
55. We’re on track to have excellent automated insights
56. anomaly detection
57. App code -> statsD -> Graphite -> Dashboards -> Insights by hand by hand by hand AUTOMATIC! AUTOMATIC!
58. (…if only we had the data)
59. How do we automate data collection?
60. currently have ganglia, New relic, collectD, etc
61. NOT WHAT WE NEED
62. they provide data about your raw machines, not your CUSTOM DEVELOPED TECHNOLOGY And Application level logic
63. Healthy servers don’t make you money. Healthy services do.
64. enter LARIMAR
65. Full disclosure: this is my new PROJECT ! we’re going to talk about it because i’m pretty excited and the beta is opening up soon.
66. LARIMAR uses raw machine metrics to infer App level architecture and inform developers about problems
67. A service: cpu resources disk io PCAP data ports Used syscalls
68. A service: cpu resources disk io PCAP data ports Used syscalls service fingerprint MACHINE LEARNING
69. A service: cpu resources disk io PCAP data ports Used syscalls ABNORMAL BEHAVIOR MACHINE LEARNING
70. a system: service service service service service MACHINE LEARNING graphical system fingerprint
71. a system: service service service service service MACHINE LEARNING Abnormal, holistic system behavior
72. Larimar automates both analysis And Relevant data collection
73. so your developers can focus on coding and acting on insights
74. No configuration!
75. App code -> statsD -> Graphite -> Dashboards -> Insights by hand AUTOMATIC! AUTOMATIC! AUTOMATIC! AUTOMATIC!
76. 1. collect 2. analyze 3. ACT 4. Profit!
77. organizational shifts are still needed to inspire ACTION on Data
78. but ACTION is easier to inspire when there is lots of data and lots of insight everywhere
79. Create a culture where your developers create these kinds of tools
80. When a data driven mindset is the default, tools will build themselves.
81. Thanks! @abestanway ! ! larimar.io @larimarhq