Machine Learning without the Hype

Machine learning is both a highly overloaded and hyped topic. This talk covers one specific area in this space — anomaly detection of time-series data. It sounds very narrow, but is widely applicable in IT security and operations.

In particular we take a look at:
* What is artificial intelligence, machine learning, and deep learning mean in general?
* When is a rule-based approach the right solution and when do you need machine learning?
* What does machine learning mean for time-series data?
* What is the difference between supervised and unsupervised learning in this area?
* What could an example with an actual dataset look like?

Transcript

1. Machine Learning ̴̴̴̴̴ohne Hype ̴̴̴̴Philipp Krenn̴̴@xeraa
2. ̴̴̴̴̴Developer
4. Machine Learning is going viral...
7. ❝Using #DeepLearning when all you needed was a few if statements. #MachineLearning #DataScience❞ —https://twitter.com/randal_olson/status/927157485240311808
11. Agenda Machine Learning Domain Dataset
12. Machine Learning
13. Artificial Intelligence Machine Learning Deep Learning !
14. https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/
15. General AI Human characteristics
16. AI Winter
17. Narrow AI Specific tasks
19. Facebook alt="Image may contain: ocean, sky, bridge, cloud, outdoor, water and nature"
22. PS: A lot of Chatbots are not AI
25. ❝Alice: I love stateless protocols! Bob: There has to be something bad about them. Alice: Bad about what?❞ —https://twitter.com/znjp/status/933405548678021120
26. Machine Learning Algorithms parse data → learn from it → make a determination or prediction "Trained" machine
27. ❝Learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.❞
28. ❝"Machine Learning is an emerging tech!" Logistic regression 1958 Hidden Markov Model 1960 Support Vector Machine 1963 k-nearest neighbors 1967 Artificial Neural Networks 1975 Expectation Maximization 1977 Decision tree 1986 Q-learning 1989 Random forest 1995❞ —https://twitter.com/farbodsaraf/status/977916871000412160
29. https://twitter.com/ algorithmia/status/ 1009486664933052416
30. ❝But saying "powered by AI" is like saying you’re "powered by the internet" or "powered by computer code". By itself, it means nothing.❞ —https://twitter.com/jensenharris/status/999119292086960128
31. Learning Regression Ranking Clustering
32. https://twitter.com/ShawnWildermuth/status/932724124237123584
34. For children and machines Watch your language
35. Statistics 101: Linear Regression
36. ❝We are leveraging machine learning.❞
37. https://twitter.com/LesGuessing/status/997146590442799105
38. Supervised Learning Input features and output labels are defined
39. Unsupervised Learning Unlabeled dataset Discover hidden relationships
40. https://xkcd.com/882/
43. Reinforcement Learning Feedback loop to optimize some parameter
44. Deep Learning Neural network producing a probability vector Lots of training and parallelization
45. https://www.youtube.com/watch?v=bxe2T-V8XRs
46. Access to a unique data set is inherently valuable
48. ❝"What's the difference between AI and ML?" "It's AI when you're raising money, it's ML when you're trying to hire people."❞ —https://twitter.com/WAWilsonIV/status/925599712849174528
49. Domain
50. Patterns Trend (stationary) Cyclical Seasonal Irregular
51. Anomaly Point Anomalies Contextual Anomalies Collective Anomalies
52. Breakouts Mean Shift Ramp Up
53. Anomaly Detection with Machine Learning Supervised Learning Unsupervised Learning
54. Examples IT operations: Spiking 500s Security analytics: Unusual DNS activity Business analytics: Rare log message
55. Visual Inspection Complex, fast moving data Humans not made to stare at graphs Easy to miss
56. Where is the Anomaly?
57. Static Rules Definition False positives & negatives Tuning and adjustment
58. Which threshold?
59. ̴̴̴̴̴̴̴̴̴̴̴̴Machine learning
60. ❝OH: "Do you run any CPU intensive application on your laptop? Like, machine learning, or Slack?" ! ❞ —https://twitter.com/jpetazzo/status/932464823530430464
61. Frameworks TensorFlow Keras SciKit ...
62. How to build ML pipelines? ETL Data storage Optimization algorithms
63. ❝I see you expected clean data. That's cute.❞
65. Model Baseline: What is normal?
68. Unsupervised
69. Evolves "Online" model learns continuously and ages out data
70. Single Time Series Example: Unusual traffic?
71. Multiple Time Series Multiple metrics or single metric split up Each series modeled independently Example: Unusual activity by country?
73. Dataset
74. nginx access log { "source": "/home/ec2-user/data/production-4/prod4elasticlog/_logs/access-logs541.log", "beat": { "hostname": "ip-172-31-5-206", "name": "ip-172-31-5-206", "version": "5.4.0" }, "@timestamp": "2017-03-08T11:44:51.562Z", "read_timestamp": "2017-06-20T08:49:58.538Z", "fileset": { "name": "access", "module": "nginx" },
75. "nginx": { "access": { "body_sent": { "bytes": "3262" }, "url": "/assets/blt1afcb054f02e257c/logo-activision.svg", "geoip": { "continent_name": "Asia", "country_iso_code": "IN", "location": { "lat": 20, "lon": 77 } },
76. "response_code": "200", "user_agent": { "device": "Other", "os_name": "Other", "os": "Other", "name": "Other" }, "http_version": "1.1", "method": "GET", "remote_ip": "192.19.197.26" } }, "prospector": { "type": "log" } }
83. Most of the internet went down
84. PS: When everything is on , nobody cares about your downloads
86. Counterfactual Reasoning Which host / IP / ... is involved in the anomaly
90. Combine Multiple Models
92. Correlation ≠ causation
93. https://xkcd.com/552/
94. https://xkcd.com/925/
95. Common problems Correlated features will mess up any model
96. Common problems Throw out most features if they are just noise
97. More features Future predictions
99. More features Clustering
100. Conclusion
101. Agenda Machine Learning Domain Dataset
102. Rules of Machine Learning: Best Practices for ML Engineering http://martin.zinkevich.org/rules_of_ml/ rules_of_ml.pdf
103. 43 rules Rule #1: Don’t be afraid to launch a product without machine learning Rule #14: Starting with an interpretable model makes debugging easier Rule #16: Plan to launch and iterate
104. Machine Learning ̴̴̴̴̴ohne Hype ̴̴̴̴Philipp Krenn̴̴@xeraa