Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Monitoring on a budget
Search
Sponsored
·
Ship Features Fearlessly
Turn features on and off without deploys. Used by thousands of Ruby developers.
→
C J Silverio
June 03, 2016
Programming
320
2
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Monitoring on a budget
How npm does monitoring & metrics on a budget.
C J Silverio
June 03, 2016
Transcript
71 slides
monitoring on a budget
a few animated gifs with the Twelfth Doctor (0 cats)
C J Silverio vp of engineering, @ceejbot
let's talk npm by the numbers
205 million packages Tuesday 10K requests/sec
npm is 25 people 4 of us run the registry
when the company was formed 5 people total
you outsource many services when you're tiny
you pull them back in-house when you succeed
success is sometimes a catastrophe
npm's scale: runaway success npm's staff: wouldn't this be neat
mission: know this on a budget
None
2 questions: is the registry up? how well is it
performing?
is the registry up? monitoring
how well is it performing? metrics
monitoring
monitoring == pull ask questions that you know the right
answers for
Is this host up? Is this cert about to expire?
Is the DB replication keeping up?
if you get the wrong answer somebody gets paged
nagios state of the art in free
None
It's okay. We never look at it. It just triggers
Pager Duty.
nagios’s virtues: reliability & custom checks
goal: never page anybody
self-healing checks automate the fix if you can!
monitoring == unit tests a ratchet for continuous improvement
external monitoring ping services
you must monitor but that's just the start
monitoring tells you what it doesn't tell you why
metrics
Q: What's a metric? A: A name + a value
+ a time.
counter: it happened N times gauge: it's Y-sized right now
rate: it's happening N times/second timing: it took X milliseconds
metrics == push the app gives you numbers
emit from a service store in timeseries db query &
graph
the usual stack statsd ➜ graphite ➜ grafana
None
None
statsd uses UDP
Q: Why not send metrics over UDP? A: You care
about receiving them.
just try to install graphite
for-pay/SAAS services exist but I can't afford them
monitoring 400 processes right now 12+ GB of log data
a day
interlude: when should you pay?
convert the £$€ cost into engineer hours/month
pay when it's cheaper than investing an engineer (be honest
about the cost)
numbat was born “How hard can it be?” I said.
None
https://github.com/ numbat-metrics numbat - powered metrics
None
npm’s stack numbat ➜ influxdb ➜ grafana
var Emitter = require('numbat-emitter'); var emitter = new Emitter({ uri:
'tcp://localhost:3333', app: 'www', }); process.emit('metric', { name: 'request.latency', value: 30 }); process.emit('metric', { name: 'disk.used.percent', value: 36 }); process.emit('metric', { name: 'login' });
so easy to emit a metric that we just do
it any time something interesting happens
4000 metrics/sec from the registry
None
None
None
None
metrics ➜ alerts
Server handling expected traffic? Latency higher than normal? Error rate
higher than usual?
metrics comprise a data stream send the stream to more
than one place!
anomaly detection
recap time!
your web apps are backed by something
what's it up to? how do you know?
get data on what your services are up to
what: monitoring yes/no questions
why: metrics data changing over time
next: anomaly detection predictions & trends
automate don't require humans
npm install -g npm@latest @ceejbot on all the things npm
loves you
More Decks by C J Silverio
See All by C J Silverio
The economics of package management
ceejbot
4
1.6k
The future of (javascript) modules (in node)
ceejbot
1
350
Keeping JavaScript safe
ceejbot
3
510
ceej's how to solve it
ceejbot
6
790
work-life balance at npm
ceejbot
5
820
hash functions and you!
ceejbot
2
400
The accidental noder
ceejbot
2
190
Design Patterns & Modularity in the npm Registry
ceejbot
3
220
Cheating Gall's Law: MediterraneaJS edition
ceejbot
4
370
Other Decks in Programming
See All in Programming
止まらない比較を止める — 余帰納法で無限ストリームを比べる
tarao
0
190
AIコーディング時代のKotest ユニットテストから APIレベルの仕様固定化テストへ
ooga04
0
170
Claude Codeのおかげでやりたいことがやれるようになったんだよォー!
sms_tech
0
100
LLM 101
dalinaum
0
500
なぜ私たちは「現時点では」KMPの導入を見送ったのか
chigichan24
1
290
LLM 101: Exploring LLMs, Starting with Tokens.
dalinaum
0
110
小さなアプリを作って公開するまで
onozaty
0
190
Rails初心者がパスキーを実装してみて大変だったこと
ktombow
0
210
Apple Intelligenceとオンデバイス時代の到来
yahir0
0
250
AIがコードを書いてくれる時代、我々は何を確かめるのか
yosuke_furukawa
PRO
13
6k
オーケストレーション記述言語 Katari
yukikurage
0
270
선언형 햅틱 라이브러리 진동을 소개합니다
l2hyunwoo
1
320
Featured
See All Featured
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
10
1.4k
Test your architecture with Archunit
thirion
2
2.5k
Rails Girls Zürich Keynote
gr2m
96
14k
AI in Enterprises - Java and Open Source to the Rescue
ivargrimstad
0
1.6k
Claude Code どこまでも/ Claude Code Everywhere
nwiizo
68
60k
Facilitating Awesome Meetings
lara
57
7.2k
A designer walks into a library…
pauljervisheath
211
25k
Making Projects Easy
brettharned
120
6.9k
The Curse of the Amulet
leimatthew05
4
17k
Building Adaptive Systems
keathley
44
3.3k
For a Future-Friendly Web
brad_frost
184
11k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
123
22k