Upgrade to Pro — share decks privately, control downloads, hide ads and more …

eBPF in Microservices Observability

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for JBD JBD
August 18, 2021
Programming
1.8k
1

eBPF in Microservices Observability

Avatar for JBD

JBD

August 18, 2021

More Decks by JBD

See All by JBD
eBPF in Microservices Observability at eBPF Day
Avatar for JBD rakyll
1
2.2k
OpenTelemetry at AWS
Avatar for JBD rakyll
1
1.9k
Debugging Code Generation in Go
Avatar for JBD rakyll
5
1.7k
Are you ready for production?
Avatar for JBD rakyll
8
2.9k
Servers are doomed to fail
Avatar for JBD rakyll
3
1.6k
Serverless Containers
Avatar for JBD rakyll
1
290
Critical Path Analysis
Avatar for JBD rakyll
0
700
Monitoring and Debugging Containers
Avatar for JBD rakyll
2
1.1k
CPDD
Avatar for JBD rakyll
0
4.3k

Other Decks in Programming

See All in Programming
新規プロダクトを高速で生み出すハーネスエンジニアリング
Avatar for Ryohei Ikegami seanchas116
15
7k
今さら聞けないCancellationToken
Avatar for tkym htkym
0
200
プラグインで拡張される Context をtype-safe にする難しさと設計判断
Avatar for kazupon kazupon
2
420
Swiftのレキシカルスコープ管理
Avatar for kntk kntkymt
0
200
Moments When Things Go Wrong
Avatar for Aurimas Liutikas aurimas
3
120
AI駆動開発で崩れていくコードベースを立て直す
Avatar for Kyoko NR kyoko_nr_nr
1
390
次世代リンターで探る、tsgo 時代における型認識カスタムルールの現実解
Avatar for Yuta Takahashi ytakahashii
3
1.2k
RTSPクライアントを自作してみた話
Avatar for simotin13 simotin13
0
300
TypeScriptだけでAIエージェントを作る フロント・エージェント・インフラのフルスタック実践
Avatar for Har1101 har1101
6
1.2k
バックエンドにElysiaJSを採用して気付いた、良い点・悪い点
Avatar for わんこ(Wanko_IT) wanko_it
1
190
OCRを使ってゲームのアイテムをデータ化する
Avatar for Kishikawa Katsumi kishikawakatsumi
0
120
AIチームを指揮するOSS「TAKT」活用術 / How to Use “TAKT,” an OSS Tool for Orchestrating AI Teams
Avatar for nrs nrslib
6
730

Featured

See All Featured
How To Speak Unicorn (iThemes Webinar)
Avatar for Michelle Schulp Hunt marktimemedia
1
470
Improving Core Web Vitals using Speculation Rules API
Avatar for Sergey Chernyshev sergeychernyshev
21
1.5k
A brief & incomplete history of 
UX Design for the World Wide Web: 1989–2019
Avatar for Jason CranfordTeague jct
2
380
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
66
8.5k
Evolving SEO for Evolving Search Engines
Avatar for Ryan Jones ryanjones
0
210
<Decoding/> the Language of Devs - We Love SEO 2024
Avatar for Nikki Halliwell nikkihalliwell
1
230
From π to Pie charts
Avatar for Rasagy Sharma rasagy
0
190
Un-Boring Meetings
Avatar for Sebastian Deterding codingconduct
0
300
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
Avatar for Akash Hashmi akashhashmi
0
350
Visual Storytelling: How to be a Superhuman Communicator
Avatar for David Neal reverentgeek
2
540
Six Lessons from altMBA
Avatar for Skipper Chong Warson skipperchong
29
4.3k
Side Projects
Avatar for Sacha Greif sachag
455
43k

Transcript

  1. @rakyll eBPF in Microservices Observability Jaana Dogan Principal Engineer, AWS

    [email protected]

  2. @rakyll About me • Not a Linux developer. • Working

    on monitoring, observability and performance. • Multi-tenancy and microservices focus.

  3. @rakyll

  4. @rakyll How does eBPF work? process JIT compiler Verifier Sockets

    TCP/IP BPF Maps code (accessible from the user space)

  5. @rakyll Where can eBPF hook into? - Kernel and user

    functions - System calls - Network events - Kernel tracepoints

  6. @rakyll Challenges in microservices

  7. @rakyll Challenges in microservices We don’t just monitor VMs or

    processes. We monitor critical paths.

  8. @rakyll What’s next? service service database storage service

  9. @rakyll What’s next? service service database storage service

  10. @rakyll Challenges in microservices Context matters. Downstream stack don’t have

    context.

  11. @rakyll What’s next? process Linux kernel process process M:N Problem

  12. @rakyll What’s next? process Linux kernel process process RPCs M:N

    Problem

  13. @rakyll What’s next? process Linux kernel process process RPCs container

    container M:N Problem

  14. @rakyll What’s next? process Linux kernel process process RPCs container

    container Kubernetes pod, ECS task M:N Problem

  15. @rakyll Challenges in microservices We initially debug RPCs. We debug

    functions or syscalls secondarily.

  16. @rakyll Challenges in microservices Too much data. Need runtime controls

    to modify the collection.

  17. @rakyll Challenges in microservices Instrumentation is a two-year roadmap. Data

    is not consistent.

  18. @rakyll Networking observability is core. Out of the box instrumentation

    is essential. Extensibility in runtime is critical. Decoration and enrichment is needed.

  19. @rakyll How does eBPF help?

  20. @rakyll Network Diagnostics TCP, UDP, HTTP, gRPC metrics Inspect protocols

    (MySQL, Postgres, ...)

  21. @rakyll Service Maps

  22. @rakyll Distributed Traces Automatically create request span if a trace

    header is present. GET /users HTTP/1.1 Host: users.service Accept-Encoding: gzip, deflate Connection: Keep-Alive Traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

  23. @rakyll Fleet-wide Profiling context kernel

  24. @rakyll Decorating with Context eBPF agent process JIT compiler Verifier

    Sockets TCP/IP BPF Maps API Server

  25. @rakyll Runtime Extensibility eBPF agent process JIT compiler Verifier Sockets

    TCP/IP BPF Maps code

  26. @rakyll Examples - Cillium/Hubble - Pixie - Flowmill

  27. @rakyll What’s next? - High level language to write probes.

    - Make eBPF agents widely available. - More platforms supporting eBPF. - Reusable eBPF event processing.

  28. @rakyll Thank you Jaana Dogan [email protected]