Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Introduction of Skyline
Search
Takumi Sakamoto
July 12, 2013
Programming
0
180
Introduction of Skyline
Takumi Sakamoto
July 12, 2013
Tweet
Share
More Decks by Takumi Sakamoto
See All by Takumi Sakamoto
Building Data Pipelines with Apache Airflow
takus
3
2.8k
The Missing Pieces of Amazon ECS (for me)
takus
2
910
マイクロサービスとSRECon / #microserv
takus
14
4.5k
Slack と Incident Command System / #slackbook
takus
2
700
Why and How SmartNews uses SaaS?
takus
2
3k
スマートニュースの世界進出を支えるログ解析基盤 #jawsdays #tech
takus
64
18k
SmartNews と Datadog
takus
6
3.9k
ぜんぶ AWS でやらないワケ
takus
55
110k
Velocity Conference 2013 Report
takus
2
1.7k
Other Decks in Programming
See All in Programming
MLH State of the League: 2026 Season
theycallmeswift
0
160
Langfuseと歩む生成AI活用推進
licux
3
300
ソフトウェアテスト徹底指南書の紹介
goyoki
1
110
CSC305 Summer Lecture 05
javiergs
PRO
0
110
Flutterと Vibe Coding で個人開発!
hyshu
1
260
Portapad紹介プレゼンテーション
gotoumakakeru
1
130
Claude Codeで実装以外の開発フロー、どこまで自動化できるか?失敗と成功
ndadayo
2
1.3k
Constant integer division faster than compiler-generated code
herumi
2
690
サーバーサイドのビルド時間87倍高速化
plaidtech
PRO
0
480
Flutter로 Gemini와 MCP를 활용한 Agentic App 만들기 - 박제창 2025 I/O Extended Seoul
itsmedreamwalker
0
150
Nuances on Kubernetes - RubyConf Taiwan 2025
envek
0
190
一人でAIプロダクトを作るための工夫 〜技術選定・開発プロセス編〜 / I want AI to work harder
rkaga
13
2.8k
Featured
See All Featured
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
KATA
mclloyd
32
14k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
780
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Become a Pro
speakerdeck
PRO
29
5.5k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.6k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Why Our Code Smells
bkeepers
PRO
338
57k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.4k
Transcript
Introduction of Skyline @takus monitoring casual #4 2013.07.12
Velocity 2013 • Web Performance & Operations Conference • Great
talks • Great people • Great culture
My LT Talk at Velocity • 3 Popular Ops Tools
in Japan • serverspec, growthforecast, fluentd • Uploaded on youtube :-( • http://www.youtube.com/watch? v=bRYuBQyG5Sw
Some Positive Feedbacks
The Most Interesting Talk • Avoiding Performance Regression at Twitter
• fight against perf regression in an automated fashion • http://ameblo.jp/principia-ca/ entry-11561132297.html
Today’s Topic
Background • Etsy deploys their app 30+ times per day
• Optimize for quick recovery by anticipating problems, instead of fearing human error • Can’t fix what you don’t measure! • If it moves, graph it
Too Many Graphs • 250,000+ dashboards • If a graph
spikes and no one is watching, does it really spike? • There are things we do not know we don’t know.
Kale • Skyline • Detect unknown anomalies • Oculus •
Detect unknown correlations http://codeascraft.com/2013/06/11/introducing-kale/
None
Architecture Horizon Horizon Horizon Redis Analyzer Manager Analyzer Analyzer Analyzer
Assign Metrics Keys Get Metrics Keys Fetch Timeseries Metrics
Horizon • Listeners • Receiving metrics & store them to
queue • Workers • Inserting metrics to redis with Messagepack • Roombas • Purging metrics in Redis at a regular interval
Analyzer • Assigning Redis keys to each process process •
decode from Messagepack • run the detection algorithm
How to Detect Anomalies? • Consensus model • If the
majority of algorithms agree, the metric will be classified as anomalous • Use your own algorithm for each application • as long as you return a boolean, you can add any sort of algorithm you like
Basic Algorithm • A metric is anomalous if its latest
datapoint is over three standard deviations above its moving average
Anomaly?
Conclusion • Etsy monitors 250,000+ graphs • If a graph
spikes and no one is watching, does it really spike? • Skyline • detects unknown anomalies • consensus with any algorithms you like