data driven development

as given at Scandinavian Developer's Conference 2012

Transcript

1. data driven development Elise Huard - Scandinavian Developer’s Conference 2012 Tuesday, 17 April 12
2. data driven development Elise Huard - Scandinavian Developer’s Conference 2012 Tuesday, 17 April 12
3. Data Tuesday, 17 April 12
4. Developers Tuesday, 17 April 12
5. Code metrics Tuesday, 17 April 12
6. Tuesday, 17 April 12
7. Logs Tuesday, 17 April 12
8. Logs = streams syslog Graphite Tuesday, 17 April 12
9. Monitoring + Notifications Tuesday, 17 April 12
10. Tuesday, 17 April 12
11. A/B testing, multivariate testing Tuesday, 17 April 12
12. Tuesday, 17 April 12
13. Tuesday, 17 April 12
14. Tuesday, 17 April 12
15. Tuesday, 17 April 12
16. Business Data Tuesday, 17 April 12
17. diagnostics trends, patterns and predictions opportunities Tuesday, 17 April 12
18. diagnostics trends, patterns and predictions opportunities Tuesday, 17 April 12
19. R Tuesday, 17 April 12
20. “The best thing about R is that it was developed by statisticians. The worst thing about R is that (...) it was developed by statisticians.” Bo Cowgill (Google Inc.) Tuesday, 17 April 12
21. Tuesday, 17 April 12
22. Data Cleaning Exploration Confirmation Interpretation Tuesday, 17 April 12
23. Data cleaning Tuesday, 17 April 12
24. format remove corrupted data (carefully remove outliers) Tuesday, 17 April 12
25. Exploration Tuesday, 17 April 12
26. > summary(mammals, digits=8) Name Body.Weight Brain.Weight Red Fox :1 Min. : 3.0 Min. : 26.0000 Pig :1 1st Qu.: 35.5 1st Qu.: 138.5000 Man :1 Median : 100.0 Median : 406.0000 Kangaroo:1 Mean : 761.2 Mean :1000.4667 Jaguar :1 3rd Qu.: 493.0 3rd Qu.: 667.5000 Horse :1 Max. :6654.0 Max. :5712.0000 (Other) :9 Tuesday, 17 April 12
27. Tuesday, 17 April 12
28. http://www.informationisbeautiful.net/visualizations/the-billion-pound-o-gram/ Tuesday, 17 April 12
29. Tuesday, 17 April 12
30. Confirmation Tuesday, 17 April 12
31. Interpretation Tuesday, 17 April 12
32. http://radar.oreilly.com/2012/03/subject-matter-experts-data-stories-analysis.html Tuesday, 17 April 12
33. A few examples Tuesday, 17 April 12
34. Timeseries Tuesday, 17 April 12
35. Tuesday, 17 April 12
36. Tuesday, 17 April 12
37. Recommendation engines Tuesday, 17 April 12
38. http://www.techdirt.com/blog/innovation/articles/20120409/03412518422/why-netflix-never-implemented-algorithm-that-won-netflix-1-million-challenge.shtml Tuesday, 17 April 12
39. Regression: Tuesday, 17 April 12
40. > coef(fitted.regression) (Intercept) Height -350.737192 7.717288 Tuesday, 17 April 12
41. Classification: Text mining Tuesday, 17 April 12
42. Tuesday, 17 April 12
43. Tuesday, 17 April 12
44. Clustering: Social graph Tuesday, 17 April 12
45. Clustering: Social graph Tuesday, 17 April 12
46. Tuesday, 17 April 12
47. Public data sets Tuesday, 17 April 12
48. Tuesday, 17 April 12
49. Tuesday, 17 April 12
50. Startups Tuesday, 17 April 12
51. Tuesday, 17 April 12
52. Tuesday, 17 April 12
53. Thank you Elise Huard http://jabberwocky.eu @elise_huard Tuesday, 17 April 12