Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Introduction of Skyline
Search
Takumi Sakamoto
July 12, 2013
Programming
220
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Introduction of Skyline
Takumi Sakamoto
July 12, 2013
Transcript
17 slides
Introduction of Skyline @takus monitoring casual #4 2013.07.12
Velocity 2013 • Web Performance & Operations Conference • Great
talks • Great people • Great culture
My LT Talk at Velocity • 3 Popular Ops Tools
in Japan • serverspec, growthforecast, fluentd • Uploaded on youtube :-( • http://www.youtube.com/watch? v=bRYuBQyG5Sw
Some Positive Feedbacks
The Most Interesting Talk • Avoiding Performance Regression at Twitter
• fight against perf regression in an automated fashion • http://ameblo.jp/principia-ca/ entry-11561132297.html
Today’s Topic
Background • Etsy deploys their app 30+ times per day
• Optimize for quick recovery by anticipating problems, instead of fearing human error • Can’t fix what you don’t measure! • If it moves, graph it
Too Many Graphs • 250,000+ dashboards • If a graph
spikes and no one is watching, does it really spike? • There are things we do not know we don’t know.
Kale • Skyline • Detect unknown anomalies • Oculus •
Detect unknown correlations http://codeascraft.com/2013/06/11/introducing-kale/
None
Architecture Horizon Horizon Horizon Redis Analyzer Manager Analyzer Analyzer Analyzer
Assign Metrics Keys Get Metrics Keys Fetch Timeseries Metrics
Horizon • Listeners • Receiving metrics & store them to
queue • Workers • Inserting metrics to redis with Messagepack • Roombas • Purging metrics in Redis at a regular interval
Analyzer • Assigning Redis keys to each process process •
decode from Messagepack • run the detection algorithm
How to Detect Anomalies? • Consensus model • If the
majority of algorithms agree, the metric will be classified as anomalous • Use your own algorithm for each application • as long as you return a boolean, you can add any sort of algorithm you like
Basic Algorithm • A metric is anomalous if its latest
datapoint is over three standard deviations above its moving average
Anomaly?
Conclusion • Etsy monitors 250,000+ graphs • If a graph
spikes and no one is watching, does it really spike? • Skyline • detects unknown anomalies • consensus with any algorithms you like
More Decks by Takumi Sakamoto
See All by Takumi Sakamoto
Building Data Pipelines with Apache Airflow
takus
3
3k
The Missing Pieces of Amazon ECS (for me)
takus
2
1k
マイクロサービスとSRECon / #microserv
takus
14
4.7k
Slack と Incident Command System / #slackbook
takus
2
800
Why and How SmartNews uses SaaS?
takus
2
3.2k
スマートニュースの世界進出を支えるログ解析基盤 #jawsdays #tech
takus
64
19k
SmartNews と Datadog
takus
6
4.1k
ぜんぶ AWS でやらないワケ
takus
55
110k
Velocity Conference 2013 Report
takus
2
1.7k
Other Decks in Programming
See All in Programming
祈るテストから、探索するテストへ 〜Lincheckではじめる並行処理テスト〜
yyh_gl
0
400
小さなアプリを作って公開するまで
onozaty
0
210
Apple Intelligenceとオンデバイス時代の到来
yahir0
0
250
AI駆動開発4年目の現在地(『あなたはFDEになるべきか?』イベント資料)
kumaiu
0
680
読みやすい!と評判(?)の発表の準備にかける私のこだわり
shoheimitani
0
110
AWS CDKを理解するために公式の定義を深掘って解説してみる / Understanding AWS CDK by Digging into Its Official Definition
akihisaikeda
3
320
LLM 101
dalinaum
0
520
疎結合とは何か?ソフトウェア設計における結合バランスについて考える #phpcon_niigata
kajitack
2
390
手の動かし方がわからなかった私が個人開発者になるまで
usagi0920_pochi
0
320
作れるエンジニアから、価値を届けるエンジニアへ
ryunakayama
1
320
Growing PicoRuby's Ecosystem on ESP32
yuuu
2
150
SODA - FACT BOOK(EN)
sodainc
1
270
Featured
See All Featured
Producing Creativity
orderedlist
PRO
348
41k
Color Theory Basics | Prateek | Gurzu
gurzu
1
530
Tell your own story through comics
letsgokoyo
1
1.1k
End of SEO as We Know It (SMX Advanced Version)
ipullrank
3
4.5k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
123
22k
Public Speaking Without Barfing On Your Shoes - THAT 2023
reverentgeek
1
670
Intergalactic Javascript Robots from Outer Space
tanoku
273
28k
How to build a perfect <img>
jonoalderson
1
6.2k
Site-Speed That Sticks
csswizardry
13
1.6k
A designer walks into a library…
pauljervisheath
211
25k
Navigating Team Friction
lara
192
16k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
1.7k