Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Monitoring in Kubernetes with Prometheus and Gr...

Monitoring in Kubernetes with Prometheus and Grafana

Kubernetes makes it easy and reliable to deploy and run your services. But in order to optimize performance and scale them, you need to know more than just that they are running. In this talk you will learn how to set up a powerful monitoring infrastructure on Kubernetes with Prometheus and Grafana which lets you measure and analyze the latency, availability and resource usage of your complete system as well as individual services and requests.

Bastian Hofmann

July 21, 2020
Tweet

More Decks by Bastian Hofmann

Other Decks in Programming

Transcript

  1. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 1

    © Copyright 2020 Rancher Labs. All Rights Reserved. 1 Monitoring in Kubernetes with Prometheus and Grafana BASTIAN HOFMANN Field Engineer - DACH
  2. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 2

    Rancher Technical Overview Rancher Technical Overview © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 5 Rancher’s recipe for production quality Kubernetes at scale Shared Tooling & Services Security & Authentication Simplified Cluster Operations & Infrastructure Management Policy management Pod & network security policies CIS benchmark monitoring RBAC policies Configuration enforcement Visibility & diagnostics Centralized audit Monitoring & alerting Kubernetes version management Node pool management Cluster provisioning Amazon EKS Azure AKS Google GKE Cloud Datacenter Dev Branch Edge Secure Application Deployment Routing Autoscaling Metrics Load Balancing Canary Git Deployments Beta © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 1 Rancher enables production quality Kubernetes operations everywhere Amazon EKS Azure AKS Google GKE Containerized App 1 Containerized App 2 Containerized App 3 Cloud Datacenter Dev Branch Edge Central Management Shared Tooling & Services Security & Authentication Simplified Cluster Operations & Infrastructure Management Policy management Pod & network security policies CIS benchmark monitoring RBAC policies Configuration enforcement Visibility & diagnostics Centralized audit Monitoring & alerting Kubernetes version management Node pool management Cluster provisioning Infrastructure agnostic
  3. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 3

    Agenda • How to set up Prometheus and Grafana • How to get and visualize metrics from Kubernetes • How to get and visualize metrics from your own applications • How to add alerts • How to collect logs from your applications • How Service Meshes help with traffic observability
  4. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 8

    Standardization of compute, network and service discovery
  5. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 11

    Custom Resource Definitions • Extend the Kubernetes API with additional Resource Definitions • Certificate • MySQLCluster • Prometheus • … • Deploy a controller into the cluster that listens on creation, change and deletion of these resources and perform the necessary actions
  6. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 14

    Manages and Configures Prometheus, Alertmanager (and Grafana)
  7. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 15

    CustomResourceDefinitions • Workload Management • Prometheus • Alertmanager • Prometheus Configuration • ServiceMonitor • PodMonitor • Rule
  8. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 22

    Several solutions • Commercial • OpenSource • ElasticSearch – FluentD/FluentBit/Logstash – Kibana • Loki – Promtail – Grafana
  9. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 25

    Service Meshes • Observability • Traffic Control • Traffic Security • Connectivity • On top of the Kubernetes Pod network
  10. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 26

    Service Meshes • Istio • Linkerd • Maesh • Kuma • …
  11. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 29

    Traffic Security • Automatic mutual TLS encryption of traffic • Automatic certificate management • Optional certificate-based authentication of traffic
  12. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 31

    Traffic Control • Advanced traffic management between services in the mesh • Blue/Green Deployments • Canary Deployments • A/B Testing • Fault injection • Circuit breakers
  13. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 33

    Multi-cluster Service Mesh • Connect separate Kubernetes cluster together • Securely handle traffic between clusters
  14. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 34

    Observability • Monitor all TCP connections, HTTP and GRCP requests • Bytes in/out • Amount of requests • Success rate • Response time • Visualize service communication
  15. © Copyright 2020 Rancher Labs. All Rights Reserved. Confidential 36

    Tracing • Trace a single incoming request through all services • Enhance with application spans through Open Tracing