Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Monitoring in Motion: Challenges of Monitoring ...
Search
Ilan Rabinovitch
February 26, 2016
Technology
0
86
Monitoring in Motion: Challenges of Monitoring Containers and Kuberntes
Ilan Rabinovitch
February 26, 2016
Tweet
Share
More Decks by Ilan Rabinovitch
See All by Ilan Rabinovitch
Monitoring in Motion - ContainerCon 2016
irabinovitch
0
90
Data Driven Post Mortems at Datadog - LinuxCon 2016
irabinovitch
1
200
Introduction to Docker Monitoring
irabinovitch
0
120
OSCON 2016 - Monitoring in Motion
irabinovitch
2
160
Monitoring OpenStack at Lithium (OpenStack Summit Austin 2016)
irabinovitch
0
58
LinuxFest Northwest 2016 - Monitoring 101
irabinovitch
0
36
Monitoring ECS and Dynamic Infrastructure
irabinovitch
0
100
Doing DevOps Right with Datadog + Pagerduty
irabinovitch
0
110
Docker Usage Patterns - Docker Meetup Palo Alto - Nov 2015
irabinovitch
0
58
Other Decks in Technology
See All in Technology
『GRANBLUE FANTASY Relink』ソフトウェアラスタライザによる実践的なオクルージョンカリング
cygames
0
180
実務における脅威モデリングを考えよう
nikinusu
0
670
株式会社EventHub・エンジニア採用資料
eventhub
0
3k
QAに対する超個人的な解釈 / Personal Take on QA
toma_sm
1
120
watsonx.ai Dojo 環境準備について
oniak3ibm
PRO
0
340
Creative UIs with Compose: DroidKaigi 2024
chrishorner
1
590
再考 アクターモデル/ reconsider actor model
ytake
0
360
アプリをリリースできる状態に保ったまま 段階的にリファクタリングするための 戦略と戦術 / Strategies and tactics for incremental refactoring
yanzm
6
1.4k
スタッフエンジニアの道: The Staff Engineer’s Path
snoozer05
PRO
44
14k
グイグイ系QAマネージャーの仕事
sadonosake
0
350
Technical Writing Meetup vol.35
soracom
PRO
2
120
AIを活用した柔軟かつ効率的な社内リソース検索への取り組み
cygames
0
190
Featured
See All Featured
Web development in the modern age
philhawksworth
205
10k
Pencils Down: Stop Designing & Start Developing
hursman
119
11k
Practical Orchestrator
shlominoach
185
10k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
38
9.2k
jQuery: Nuts, Bolts and Bling
dougneiner
61
7.4k
Facilitating Awesome Meetings
lara
49
6k
GraphQLの誤解/rethinking-graphql
sonatard
65
9.8k
The Straight Up "How To Draw Better" Workshop
denniskardys
230
130k
Making the Leap to Tech Lead
cromwellryan
128
8.8k
The Power of CSS Pseudo Elements
geoffreycrofte
71
5.3k
What's in a price? How to price your products and services
michaelherold
242
11k
4 Signs Your Business is Dying
shpigford
179
21k
Transcript
Monitoring In Motion Challenges in Monitoring Kubernetes & Containers Cloud
Native SF Meetup Feb 25, 2016 Ilan Rabinovitch Director, Community Datadog
About Me • Long time Datadog user. • Prior to
Datadog built automation and monitoring tooling at Ooyala and Edmunds.com • SCALE and TXLF Co-Founder Ilan Rabinovitch Datadog
[email protected]
@irabinovitch
Agenda • Monitoring 101 - Crash Course • Challenges in
Monitoring Dynamic Infrastructure • Demo Time • Questions?
Monitoring Everything
None
@honest_update on Twitter
Quick Overview of Datadog • Monitoring for modern applications. •
Time series storage of metrics and events. • Trending, alerting and anomaly detection. • Hundreds of integrations out of the box.
Monitoring 101: Categorization More at: http://goo.gl/t1Rgcg
None
Monitoring 101: Focus on symptoms More at: http://goo.gl/t1Rgcg
Recurse until you find root cause. More at: http://goo.gl/t1Rgcg
Container Monitoring Challenges
https://www.datadoghq.com/docker-adoption/
None
None
Operational Complexity •Average containers per host: N (N=4, 10/2015) •N-times
as many “hosts” to manage •Affects everything
Operational Complexity: Scale 100 instances 400 containers
Operational Complexity: Scale 160 metrics per host 640 metrics per
host
Operational Complexity: Scale 100 instances 64,000 metrics
None
Host Centric vs Service Centric
Host Centric vs Service Centric
Query Based Monitoring … … …
•Use tags, labels, etc on your hosts and metrics. •Pull
in existing labels from your infrastructure (Region, Docker Images, K8S Tags..) Query Based Monitoring By using tags, auto-adapt!
Where is my application running ? What’s the total throughput
of App X ? What’s its response time per tag ? (pod, version, DC) What’s the distribution of 5xx from Nginx per pod ?
Auto Discovery
Docker API Kubelet API Monitoring Agent Container A O A
O A O Application Container Off-The-Shelf Application (Redis, PostgreSQL, …) Containers List Metadata Additional Metadata (Pod names, RC, …) Config Backend Integration Configurations Host Level Metrics
Some Pictures Dashboards and Metrics Alerts Sharing
Demo time