eBPF in Microservices Observability at eBPF Day

@rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS
[email protected]

@rakyll About me • Not a Linux developer. • Working
on monitoring, observability and performance. • Multi-tenancy and microservices focus.

@rakyll

@rakyll How does eBPF work? process JIT compiler Verifier Sockets
TCP/IP BPF Maps code (accessible from the user space)

@rakyll Where can eBPF hook into? - Kernel and user
functions - System calls - Network events - Kernel tracepoints

@rakyll Challenges in microservices

@rakyll Challenges in microservices We don’t just monitor VMs or
processes. We monitor critical paths.

@rakyll What’s next? service service database storage service

@rakyll Challenges in microservices Context matters. Downstream stack don’t have
context.

@rakyll What’s next? process Linux kernel process process M:N Problem

@rakyll What’s next? process Linux kernel process process RPCs M:N
Problem

@rakyll What’s next? process Linux kernel process process RPCs container
container M:N Problem

@rakyll What’s next? process Linux kernel process process RPCs container
container Kubernetes pod, ECS task M:N Problem

@rakyll Challenges in microservices First, we debug the path of
the request. We debug functions or syscalls secondarily.

@rakyll Challenges in microservices Too much data. Need runtime controls
to modify the collection.

@rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data
is not consistent.

@rakyll Recap Out of the box instrumentation is critical. Networking
observability is essential. Extensibility in runtime is needed. Decoration and enrichment is needed.

@rakyll How does eBPF help?

@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols
(MySQL, Postgres, ...)

@rakyll Service Maps

@rakyll Distributed Traces Automatically create trace span if a trace
header is present. Your job is to generate and propagate the header. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

@rakyll Continuous Profiling

@rakyll Extensibility

@rakyll Decorating with Context eBPF agent process JIT compiler Verifier
Sockets TCP/IP BPF Maps API Server

@rakyll Decorating with Context

@rakyll Several projects... - Cillium/Hubble - Pixie - Flowmill -
Prodfiler - Parca

@rakyll What’s next? - High level language? - More platforms
supporting eBPF? - Reusable eBPF event processors? - Signed programs?

@rakyll Thank you Jaana Dogan [email protected]

@rakyll After party? Pixie KubeCon Happy Hour hosted by New
Relic Tomorrow, 8:30 PM PDT RSVP: pixiehh.eventbrite.com

@rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets
TCP/IP BPF Maps code

eBPF in Microservices Observability at eBPF Day

eBPF in Microservices Observability at eBPF Day

JBD

More Decks by JBD

Other Decks in Programming

Featured

Transcript

@rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS

@rakyll About me • Not a Linux developer. • Working

@rakyll

@rakyll How does eBPF work? process JIT compiler Verifier Sockets

@rakyll Where can eBPF hook into? - Kernel and user

@rakyll Challenges in microservices

@rakyll Challenges in microservices We don’t just monitor VMs or

@rakyll What’s next? service service database storage service

@rakyll What’s next? service service database storage service

@rakyll Challenges in microservices Context matters. Downstream stack don’t have

@rakyll What’s next? process Linux kernel process process M:N Problem

@rakyll What’s next? process Linux kernel process process RPCs M:N

@rakyll What’s next? process Linux kernel process process RPCs container

@rakyll What’s next? process Linux kernel process process RPCs container

@rakyll Challenges in microservices First, we debug the path of

@rakyll Challenges in microservices Too much data. Need runtime controls

@rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data

@rakyll Recap Out of the box instrumentation is critical. Networking

@rakyll How does eBPF help?

@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols

@rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols

@rakyll Service Maps

@rakyll Distributed Traces Automatically create trace span if a trace

@rakyll Continuous Profiling

@rakyll Extensibility

@rakyll Decorating with Context eBPF agent process JIT compiler Verifier

@rakyll Decorating with Context

@rakyll Several projects... - Cillium/Hubble - Pixie - Flowmill -

@rakyll What’s next? - High level language? - More platforms

@rakyll Thank you Jaana Dogan [email protected]

@rakyll After party? Pixie KubeCon Happy Hour hosted by New

@rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets