ElixirConf 2016 - Measuring your elixir App

Transcript

1. Measuring your Elixir application Renan Ranelli
2. Senior software engineer @ Milhouse (@renanranelli)
3. Milhouse (@renanranelli)
6. Why should I care?
7. Why should I care? We write code.
8. Why should I care? We write code. We do it because it generate business value.
9. Why should I care? We write code. We do it because it generate business value. But this business value is generated only when our code *runs*. *NOT* when it is written
10. Why should I care? In order to make good decisions about something, we must understand it.
11. Why should I care? In order to make good decisions about something, we must understand it. (And we want to make good decisions because we like getting paid)
12. But... there is a problem
13. Why should I care? Its hard to reason about code
14. Which one is faster?
15. We don’t know
16. Which is faster?
17. The mental model we have of our system is *always* flawed.
18. But wait! There's still hope.
19. When we measure stuff, the confusion fades:
20. When we measure stuff, the confusion fades:
21. This is *SCIENCE* !
22. This is *SCIENCE* !
23. Types of metrics & how we look at them
24. The metrics we’re interested are **time series**:
25. The metrics we’re interested are **time series**:
26. Metrics – average value
27. Metrics – average value
28. Metrics – average value ← Time
29. Metrics – average value First thing you should ask: Which time period does this average refer to ??? Averages completely destroy the **variability** and **recency**.
30. Metrics – average value A common thing to do is add a “decay” component to the average calculation:
31. Metrics – average value A common thing to do is add a “decay” component to the average calculation:
32. Metrics – distribution
33. Metrics – distribution
34. Histograms Histograms are like averages on steroids. As such, you **have** to pay attention to The aggregation period it refers to
35. Metrics – resolution
36. Metrics – resolution
37. How do I get started in the real world?
38. You have to solve these parts: ❑ Collect ❑ Store ❑ Query, Analyze & Transform ❑ Visualize ❑ … ❑ MAKE DECISIONS (!)
39. Our metrics collection architecture Log parsing Exometer & Elixometer Influxdb plugin “report api” Query & Visualize Transform & Analyze STORE Collect Store View
40. Our (boring) choices ❑ Collect •Exometer & Elixometer are very easy to set up •We have used collectd extensively in the past ❑ Store (InfluxDB) •**Very** easy to set up & configure. And free. •Built-in retention policy configuration •Awesome query language & Continuous queries •Plugs seamlessly with grafana
41. Our (boring) choices ❑ View (Grafana) •We have used it in the past. •It is awesome and beautiful (you will see!)
42. Our (boring) choices ❑ You might ask: ❑ Why you’re not using solution like Splunk, DataDog, etc?
43. Our (boring) choices ❑ You might ask: Why you’re not using solution like Splunk, DataDog, etc? ❑ Our answer: ❑ We don't like our data outside our walls. Also, NDAs and legal obligations
44. Exometer & Elixometer ❑ Exometer Exometer Reporter Metrics buffer (pre aggregated ) High frequency Metric reports Store & update Aggregated view Collect & convert Low frequency Metrics formated for other backends
45. Exometer & Elixometer ❑ This design is almost omnipresent in metrics world. ❑ Other tools like statsd, telegraf, etc might fit the role of exometer in your case. Use the one you’re most comfortable with. ❑ It is **very** important to be able to control the frequency which your storage receives requests. You don’t want to overload it.
46. DEMONSTRATION • Measuring all requests to a phoenix app Step by step instructions here: http://milhouseonsoftware.com/2016/05/08/me asuring-your-elixir-application/
47. Other things you should measure
48. Other things you should measure ❑ The Erlang VM is **awesome** (and that’s why we’re here ;P) ❑ There are lots of resources out there on how to measure, monitor and operate it. ❑ But sadly I don’t have enough time to go in depth about it ~ Use vm_stats to gather virtual machine metrics! https://github.com/ferd/vmstats
49. Other things you should measure ❑ The Erlang VM is **awesome** (and that’s why we’re here ;P) ❑ ❑ There are lots of resources out there on how to measure, monitor and operate it. ❑ ❑ But sadly I don’t have enough time to go in depth about the things the instrumentation the VM exposes ~
50. Other things you should measure ❑ The Erlang VM is **awesome** (and that’s why we’re here ;P) ❑ ❑ There are lots of resources out there on how to measure, monitor and operate it.
51. Closing remarks
52. Closing remarks ❑ Find “seams” in your code where you can “plug” measurements. ❑ Try not to spread metrics reporting “everywhere”. It’s best to separat “high level” measurements from “low level” ones. ❑ This is just the beginning. After you have this infrastructure in place, you have to start doing analysis, correlation and other math stuffs. That’s where the real *gold* is.
53. Closing remarks ❑ Bend your metrics to generate alerts. This is **really** awesome. ❑ Another (similar) metrics collection architecture is described here: http://tech.footballaddicts.com/blog/gathering-metrics-in-elixir- applications ❑ Zed Shaw has an awesome post about statistics: ❑https://zedshaw.com/archive/programmers-need-to-learn-statistics- or-i-will-kill-them-all/
54. Obrigado!
55. milhouseonsoftware.com @renanranelli /rranelli Milhouse @ (We're hiring!!!)