From New Cluster to Insight. Deploying Monitoring and Logging to Kubernetes

The question that most people ask after spinning up their first Kubernetes cluster is "how do I do monitoring and logging".

In this session we'll utilize open source tools like Prometheus, Helm, Grafana, and Loki to quickly go from being in the dark to having full visibility into the happenings of our clusters and workloads. We'll deploy an entire monitoring and logging stack to a fresh cluster live and discuss the pros and cons of such tools.

You'll leave this session with a playbook on how to bootstrap your cluster observability and some gotchas and lesson's learned from our monitoring of large workloads at DigitalOcean.

https://youtu.be/sR3jxSXY7A4

Transcript

1. digitalocean.com From New Cluster to Insight: Deploying Monitoring and Logging to Kubernetes Eddie Zaneski @eddiezane
5. digitalocean.com https://landscape.cncf.io
8. digitalocean.com Monitoring
9. digitalocean.com Things to monitor • Nodes • Pods • kube-system ◦ kube-apiserver ◦ kube-scheduler ◦ etc(d)... • APM ◦ Requests per second ◦ Error rates • App specific metrics ◦ Top selling product
12. https://coreos.com/blog/the-prometheus-operator.html
14. • 4.5 million metrics samples a second
16. digitalocean.com Logging
17. digitalocean.com Things to log • All the things
18. Shippers
19. Ingesters
23. digitalocean.com • rsyslog • 100% availability ingesting • 30k log lines ingested a second • 10+ regional aggregators
24. digitalocean.com Demo
25. digitalocean.com Thanks! @eddiezane https://github.com/eddiezane/kubernetes-observability-example https://do.co/c