Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Monitoring in Motion: Challenges of Monitoring ...
Search
Ilan Rabinovitch
February 26, 2016
Technology
0
90
Monitoring in Motion: Challenges of Monitoring Containers and Kuberntes
Ilan Rabinovitch
February 26, 2016
Tweet
Share
More Decks by Ilan Rabinovitch
See All by Ilan Rabinovitch
Monitoring in Motion - ContainerCon 2016
irabinovitch
0
92
Data Driven Post Mortems at Datadog - LinuxCon 2016
irabinovitch
1
200
Introduction to Docker Monitoring
irabinovitch
0
130
OSCON 2016 - Monitoring in Motion
irabinovitch
2
170
Monitoring OpenStack at Lithium (OpenStack Summit Austin 2016)
irabinovitch
0
59
LinuxFest Northwest 2016 - Monitoring 101
irabinovitch
0
37
Monitoring ECS and Dynamic Infrastructure
irabinovitch
0
100
Doing DevOps Right with Datadog + Pagerduty
irabinovitch
0
120
Docker Usage Patterns - Docker Meetup Palo Alto - Nov 2015
irabinovitch
0
59
Other Decks in Technology
See All in Technology
10分で学ぶKubernetesコンテナセキュリティ/10min-k8s-container-sec
mochizuki875
3
380
NW-JAWS #14 re:Invent 2024(予選落ち含)で 発表された推しアップデートについて
nagisa53
0
270
株式会社ログラス − エンジニア向け会社説明資料 / Loglass Comapany Deck for Engineer
loglass2019
3
32k
マルチプロダクト開発の現場でAWS Security Hubを1年以上運用して得た教訓
muziyoshiz
3
2.6k
Fanstaの1年を大解剖! 一人SREはどこまでできるのか!?
syossan27
2
180
DUSt3R, MASt3R, MASt3R-SfM にみる3D基盤モデル
spatial_ai_network
2
220
Server-Side Engineer of LINE Sukimani
lycorp_recruit_jp
0
330
[Ruby] Develop a Morse Code Learning Gem & Beep from Strings
oguressive
1
190
オプトインカメラ:UWB測位を応用したオプトイン型のカメラ計測
matthewlujp
0
190
Working as a Server-side Engineer at LY Corporation
lycorp_recruit_jp
0
340
ハイテク休憩
sat
PRO
2
170
UI State設計とテスト方針
rmakiyama
3
770
Featured
See All Featured
Designing Experiences People Love
moore
138
23k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
44
9.3k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
28
2.1k
Why Our Code Smells
bkeepers
PRO
335
57k
GitHub's CSS Performance
jonrohan
1031
460k
The Power of CSS Pseudo Elements
geoffreycrofte
73
5.4k
Art, The Web, and Tiny UX
lynnandtonic
298
20k
Speed Design
sergeychernyshev
25
670
Thoughts on Productivity
jonyablonski
68
4.4k
Put a Button on it: Removing Barriers to Going Fast.
kastner
59
3.6k
Typedesign – Prime Four
hannesfritz
40
2.4k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
191
16k
Transcript
Monitoring In Motion Challenges in Monitoring Kubernetes & Containers Cloud
Native SF Meetup Feb 25, 2016 Ilan Rabinovitch Director, Community Datadog
About Me • Long time Datadog user. • Prior to
Datadog built automation and monitoring tooling at Ooyala and Edmunds.com • SCALE and TXLF Co-Founder Ilan Rabinovitch Datadog
[email protected]
@irabinovitch
Agenda • Monitoring 101 - Crash Course • Challenges in
Monitoring Dynamic Infrastructure • Demo Time • Questions?
Monitoring Everything
None
@honest_update on Twitter
Quick Overview of Datadog • Monitoring for modern applications. •
Time series storage of metrics and events. • Trending, alerting and anomaly detection. • Hundreds of integrations out of the box.
Monitoring 101: Categorization More at: http://goo.gl/t1Rgcg
None
Monitoring 101: Focus on symptoms More at: http://goo.gl/t1Rgcg
Recurse until you find root cause. More at: http://goo.gl/t1Rgcg
Container Monitoring Challenges
https://www.datadoghq.com/docker-adoption/
None
None
Operational Complexity •Average containers per host: N (N=4, 10/2015) •N-times
as many “hosts” to manage •Affects everything
Operational Complexity: Scale 100 instances 400 containers
Operational Complexity: Scale 160 metrics per host 640 metrics per
host
Operational Complexity: Scale 100 instances 64,000 metrics
None
Host Centric vs Service Centric
Host Centric vs Service Centric
Query Based Monitoring … … …
•Use tags, labels, etc on your hosts and metrics. •Pull
in existing labels from your infrastructure (Region, Docker Images, K8S Tags..) Query Based Monitoring By using tags, auto-adapt!
Where is my application running ? What’s the total throughput
of App X ? What’s its response time per tag ? (pod, version, DC) What’s the distribution of 5xx from Nginx per pod ?
Auto Discovery
Docker API Kubelet API Monitoring Agent Container A O A
O A O Application Container Off-The-Shelf Application (Redis, PostgreSQL, …) Containers List Metadata Additional Metadata (Pod names, RC, …) Config Backend Integration Configurations Host Level Metrics
Some Pictures Dashboards and Metrics Alerts Sharing
Demo time