Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
production: an owner's manual
Search
Igor Wiedler
April 23, 2018
Programming
210
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
production: an owner's manual
from exec(ut) 2018
Igor Wiedler
April 23, 2018
Transcript
68 slides
production: an owner's manual
hello!
broken computers
None
getting sidetracked now so sorry* * not sorry
None
None
None
back to serious business
!
None
a production system is a system that serves real users
the goal of operations is to ensure services are reliable
in order to provide a good user experience
None
failure
app
app linux kernel cpu dram disk network power supply switches
load balancer dns submarine cables routers fiber
app linux kernel the cloud
None
• cosmic rays • disk failure • power outages •
software bugs • ...
entropy
None
capacity
None
None
None
cascading failure
None
system design
redundancy
"
scale
None
"
p1 m3 c1 m2 m1 p2 c2
data storage
"
"
protocols
None
monitoring
many components many req/s
None
measure all the things?
✅ ⏱
golden signals • latency • traffic • errors • saturation
golden signals • latency • traffic • errors • saturation
golden signals • latency • traffic • errors • saturation
golden signals • latency • traffic • errors • saturation
golden signals • latency • traffic • errors • saturation
0 - 50 [1620]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ (74.55%) 50 - 100 [ 447]: ∎∎∎∎∎∎∎∎∎∎ (20.57%) 100 - 150 [ 49]: ∎ (2.25%) 150 - 200 [ 15]: (0.69%) 200 - 250 [ 15]: (0.69%) 250 - 300 [ 10]: (0.46%) 300 - 350 [ 6]: (0.28%) 350 - 400 [ 1]: (0.05%) 400 - 450 [ 0]: (0.00%) 450 - 500 [ 4]: (0.18%)
golden signals • latency • traffic • errors • saturation
saturation traffic latency errors
None
humans
None
oops, deleted the database
bad human!
why does this button even exist?
app linux kernel cpu dram disk network power supply switches
load balancer dns submarine cables routers fiber
app linux kernel cpu dram disk network power supply switches
load balancer dns submarine cables routers fiber humans
app linux kernel cpu dram disk network power supply switches
load balancer dns submarine cables routers fiber humans h u m a n s
epic failure is almost always systemic
failure
recap
• a production system serves real users • users like
things that work and are fast • epic failure is almost always systemic
thx @igorwhilefalse
None
More Decks by Igor Wiedler
See All by Igor Wiedler
Redis Bedtime Stories
igorw
1
410
Wide Event Analytics (LISA19)
igorw
4
970
a day in the life of a request
igorw
0
190
The Power of 2
igorw
0
370
LISP 1.5 Programmer's Manual: A Dramatic Reading
igorw
0
520
The Moral Character of Software
igorw
1
340
interdisciplinary computing (domcode)
igorw
0
340
miniKanren (clojure berlin)
igorw
1
360
End the war on tabs (phpnw14)
igorw
1
1.1k
Other Decks in Programming
See All in Programming
例外からRich Errorsまで Kotlinエラーハンドリングの進化を読み解く
shiita0903
0
240
どうしても Eloquent でリポジトリパターンをやりたいんだ #phpcon_niigata / phpcon-niigata-2026
shogogg
0
670
自宅サーバはどこまで再現できるのか - Proxmox VE上のVM環境をIaCとバックアップで戻してみた
youto
0
450
ローカルLLMの基本と近況 2611 / Local LLM basics and recent situation 2611
kishida
3
660
あなたはMCPの機能を、まだ知らない。
4geru
1
740
高速開発を止めずに 性能を上げる Flyleのシステム刷新と改善の歴史
azihsoyn
0
100
[Kotlin Fest 2026] eBPF Tracing for Kotlin
takehaya
1
400
TS 7.0 で baseUrl が使えなくなった今、パスエイリアスをどうするか
bicstone
2
810
高橋メソッドでCVE解説シリーズ 〜CVE-2026-112816〜
hakkadaikon
0
270
Claude Code ハーネスDeep Dive – 君はどこまで制御する? -
takateeen_knj
6
6.3k
LLM 101: Exploring LLMs, Starting with Tokens.
dalinaum
0
120
AppFunctionsの内部実装を紐解く
hiroaki404
0
120
Featured
See All Featured
Why You Should Never Use an ORM
jnunemaker
PRO
61
10k
Done Done
chrislema
187
17k
GitHub's CSS Performance
jonrohan
1033
470k
The Straight Up "How To Draw Better" Workshop
denniskardys
239
140k
技術選定の審美眼(2025年版) / Understanding the Spiral of Technologies 2025 edition
twada
PRO
121
130k
Fireside Chat
paigeccino
43
4.1k
Code Reviewing Like a Champion
maltzj
529
40k
Impact Scores and Hybrid Strategies: The future of link building
tamaranovitovic
0
490
How Software Deployment tools have changed in the past 20 years
geshan
3
35k
Building a Scalable Design System with Sketch
lauravandoore
464
34k
jQuery: Nuts, Bolts and Bling
dougneiner
66
8.7k
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
aleyda
2
12k