Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Prometheus as exposition format for eBPF progra...

Prometheus as exposition format for eBPF programs on k8s

Nowadays every application exposes their metrics via an HTTP endpoint readable by using Prometheus. Recently the exposition format got included into the OpenMetrics standard of the CNCF. Nevertheless, this very common pattern by definition only expose metrics regarding the specific applications being observed.
This talk wants to expose the idea, and a reference implementation, of a slightly different use case that uses eBPF programs, running on Kubernetes via a Custom Resource Definition (CRD), as a source of information to allow the exposition and collection of kernel and application probes via a Prometheus endpoint.

Leonardo Di Donato

February 27, 2019
Tweet

More Decks by Leonardo Di Donato

Other Decks in Programming

Transcript

  1. OBSERVABILITY insights deductions ongoing blackbox data fine-grained MONITORING on just

    any data garbage Data lake Another really cool buzzword is TRACING. Execution path along the code. Impacts runtime performances. Usually disabled or sampled.
  2. New metric types • counter • gauge • histogram •

    summary • untyped → unknown • state set • info • gauge histogram Format • • • Exemplars A normal sample line but without the metric name. A space after the value (or timestamp if present), a hash sign, a space and then the exemplar. Histogram buckets can have them!
  3. Prometheus OpenMetrics # TYPE foo histogram foo_bucket{le="0.01"} 0 foo_bucket{le="0.1"} 8

    # {} 0.054 foo_bucket{le="1"} 10 # {id="9856e"} 0.67 foo_bucket{le="10"} 17 # {id="12fa8"} 9.8 1520879607.789 ...
  4. # HELP test_packets Data coming from packets BPF map #

    TYPE test_packets counter test_packets{key="00002",node="127.0.0.1"} 1 test_packets{key="00006",node="127.0.0.1"} 551 test_packets{key="00008",node="127.0.0.1"} 1 test_packets{key="00017",node="127.0.0.1"} 15930 test_packets{key="00089",node="127.0.0.1"} 9 test_packets{key="00233",node="127.0.0.1"} 1 bfptools/kube-bpf