The Hardest Problem in Data

Transcript

1. The Hardest Problem in Data Ronnie Chen @rondoftw Data Engineering Slack 1 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
2. 2 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
3. → Machine learning → Predictive modeling → Neural networks → Artificial intelligence 3 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
4. Counting ?! 4 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
5. 5 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
6. A simple counting problem 6 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
7. The Rules: 1. Only one number 2. Convince me it's correct 7 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
8. How many friends do you have? 8 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
9. Will I get the same number if... !"#$ I ask every person you know if they consider you their friend? 9 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
10. Will I get the same number if... ! " I ask every person that knows you if they think you would consider them a friend? 10 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
11. Is this the number of people that you'd tell a secret to? 11 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
12. But it depends!! 12 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
13. How many users do we have? 13 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
14. SELECT COUNT(*) FROM prod.users 14 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
15. user_id name email deleted 1 Alice alice@*** 2 Bob bob@*** true 3 Carol 15 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
16. SELECT COUNT(*) FROM prod.users WHERE deleted != true AND email != null 16 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
17. SELECT COUNT(*) FROM prod.users WHERE last_active > 2017-07-24 17 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
18. user_id email 12334 rchen@slack-corp.com 38602 ronnieftw@gmail.com 52981 ronnieftw@gmail.com 67640 hello@ronniechen.com 18 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
19. ¯\_(ϑ)_/¯ 19 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
20. What are you not even aware of? 20 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
21. Okay, I get it. But what's the big deal? 21 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
22. 26% of professional computing jobs were held by women in 2016 22 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
23. 23 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
24. Numbers give you authority and the appearance of objectivity 24 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
25. Counting is power. 25 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
26. 26 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
27. Counts can determine funding, set agendas, and shift priorities 27 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
28. Machine learning is like money laundering for bias — Maciej Cegłowski, founder of @Pinboard 28 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
29. 29 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
30. 30 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw
31. What you count determines what is important. 31 — WriteSpeakCode 2017 | Ronnie Chen @rondoftw